CC BY
66
Review article UDK:
37.012
Received: October, 21.2022.
Revised: December, 06.2022. d 10.23947/2334-8496-2022-10-3-177-183
Accepted: December, 09.2022. ^ CheQk (or updates
Quasi-Experimental Research as An Epistemological-Methodological Approach in
Education Research
Jelena S. Osmanovic Zajic1' , Jelena 2. Maksimovic1
'University of Nis, Faculty of Philosophy, Serbia, e-mail: jelena.osmanovic.zajic@filfak.ni.ac.rs, jelena.maksimovic@filfak.ni.ac.rs
Abstract: Pedagogy as a social-humanistic science that deals with upbringing and education expresses its complexity through a plurality of ideas, understandings and opportunities for studying, learning and researching educational practice. The methodology dealt with by experimental and quasi-experimental pedagogy is exact and serves to establish reliable data, i.e., it searches for facts and evidence that can replace research hypotheses. A synonym for quasi-experimental pedagogy is "new pedagogy^' because the evaluation of research quality in educational practice can be defined through the methodological basis of research, originality, novelty and significance. The question arises: how many quasi-experiments have been conducted so far that researchers are not even aware of, that is, they have not called this type of research by its real name? The main goal of theoretical research is to analyze the crucial differences between non-experimental, experimental and quasi-experimental research. From the presented goal of the research arises the research task which is reflected in seeking to affirm quasi-experimental research in pedagogy with the aim of bringing closer to the scientific public the characteristics and advantages of quasi-experiments in terms of streamlining implementation and practicality in the natural school environment. The paper uses the theoretical analysis method with the content analysis technique. The authors presented conclusions about the importance of quasi-experimental research as a special epistemological-methodological approach in determining the causality of educational phenomena.
Keywords: quasi-experimental pedagogy, causality, epistemological-methodological approaches, methodology of pedagogy, research plurality.
Introduction
Ontology, epistemology, and methodology are essential segments of every branch of science, including pedagogy. Because of them, it is possible to determine the criteria for the systematization of scientific knowledge and to review the state and scope of theory and pedagogy as a science. Linking research and philosophical traditions helps researchers to clarify the theoretical frameworks of their research. The research framework includes beliefs about the nature of reality (ontology), the theory of knowledge that characterizes the research (epistemology), and how that knowledge can be acquired (methodology). All these represent different directions that can be taken in research. The use of quantification for the presentation of research results must be in compliance with positivist epistemology, and for this reason, the epistemological-methodological peculiarities of education research must be understood first. Is this really the way it is in practice? The goal of this paper is to provide a brief overview of research paradigms in education research and place quasi-experimental pedagogy among applied pedagogical research used in the examination of educational practice.
The complexity of the educational process and the very nature of educational problems require the integration and combination of qualitative and quantitative methodology in research. The basic characteristics of these methodologies point to the need of conducting mixed methods research. Quantitative methodology relies on large samples in order to analyze certain parts of the population. On the other hand, qualitative methodology is based on small samples that are thoroughly described and analyzed. Combining them ensures the analysis of individual parts of the population, but also the understanding of their full complexity. Using the mixed method implies that the researcher is well acquainted the basics of both qualitative and quantitative methodologies. Finally, the concept of quasi-experimental pedagogy represents a set of logically related assumptions, concepts or propositions that
'Corresponding author: jelena.osmanovic.zajic@filfak.ni.ac.rs
Tg I © 2022 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.Org/licenses/by/4.0/).
guide research. Without determining the paradigm as the first step, there is no basis for further choices in terms of the methodology, methods, literature and research design itself. Certainly, there are problems in pedagogy related to the methodology based on the positivist paradigm, as well as those related to the so-called humanistic methodology. But what we can predict with great certainty is that pedagogy, or any other social-humanistic science, will not develop within the framework of only one methodological approach. In relation to that, the authors highlight that one can rely on the logical-methodological and, among other things, the epistemological setting as the basic philosophical starting point of research, but also that one can follow modern research trends and use a plurality of ideas when it comes to experimental and quasi-experimental pedagogy.
Epistemological-Methodological Approaches in Education Research in the Context
of Quasi-Experimental Pedagogy
During the 1980s, many quantitatively and qualitatively oriented researchers claimed that their approach was superior. Some of these researchers were "purists," as they argued that the two approaches cannot be used together due to differences in worldviews or philosophical orientations (Antwi and Hamza, 2015). Within the social and humanistic sciences, there is a need for a planned and systematized approach to basic and principled epistemological questions. Epistemology and methodology go together - they are intertwined concepts and are always in a dialectical unity, which leads to new knowledge in pedagogy. In terms of their philosophical foundations, epistemology is closer to logic and gnoseology, while methodology is focused on empiricism, i.e., "practical activity" (Sladoje Bosnjak, 2019; 2018a; 2018b).
Al-Ababneh (2020) gives the following definitions: 1. Epistemology is the theory of knowledge embedded in the theoretical perspective, and therefore in the methodology; 2. Theoretical perspective is the philosophical stance informing the methodology and thus giving a context for the process and grounding its logic and criteria; 3. Methodology is the strategy, action plan, process or design that lies behind the choice and use of certain methods and connects the choice and use of methods with the desired outcomes; 4. Methods are defined as the techniques or procedures used to gather and analyze data related to a research question or hypothesis.
Ontology refers to the nature of our beliefs about reality. Researchers have assumptions about it. The ontological question that prompts the researcher to think about is what kinds of reality exist: a single, verifiable reality or a multiple social reality. In its origin, ontology refers to a branch of philosophy that deals with articulating the nature and structure of the world. It specifies the form and nature of reality and what can be known about it. There are two broadly contrasting positions - objectivism and constructionism. Objectivism holds that there is an independent reality and constructionism assumes that reality is a product of social processes (Antwi and Hamza, 2015).
Epistemology refers to a branch of philosophy that studies the nature of knowledge and the process by which knowledge is acquired and confirmed. It represents a concern about the nature and form of knowledge, how it can be acquired and how it can be communicated to others. Epistemological questions lead the researcher to think about possibilities, objectivity, subjectivity, validity and generalization in some researched studies. Adhering to an ontological belief system, we can certainly arrive at certain epistemological assumptions. If it is assumed that there is a single verifiable truth, we strive with our beliefs to discover it and to learn "exactly how things are" and "how things really work". On the other hand, believing there is a socially constructed multiple reality leads the researcher to include other subjects and understand phenomena from a wide variety of contexts. Epistemology represents a distinctive "worldview" (Antwi and Hamza, 2015; Al-Ababneh, 2020).
Research methodology is a theoretically based approach to the study of data. Methodology can also be defined as a method used in conducting research and it reflects the question of how knowledge can be acquired. The methodological aspect of research must be in compliance with ontological and epistemological attitudes (Antwi and Hamza, 2015; Kamal, 2019). It refers to the study and critical analysis of data acquisition techniques. Methodology is the strategy, action plan, process or design that informs the choice of research methods. It guides the researcher in deciding which type of data and instruments to use in a specific study, and which tools are most suitable for researching a particular phenomenon. Finally, methods are specific means of collecting and analyzing data. Which methods will be used in a study depends on its design and the researcher's "theoretical" way of thinking. It should be noted that the use of certain methods does not entail ontological and epistemological assumptions (Rehman and Alharthi, 2016).
Ontology, epistemology and methodology are essential segments of every branch of science,
including pedagogy. Because of them, it is possible to determine the criteria for the systematization of scientific knowledge and to review the state and scope of theory and pedagogy as a science. Linking research and philosophical traditions helps researchers to clarify the theoretical frameworks of their research. The research framework includes beliefs about the nature of reality (ontology), the theory of knowledge that characterizes the research (epistemology), and how that knowledge can be acquired (methodology).
All of these are different research directions that can be taken in experimental and quasi-experimental pedagogy research. This section provided a brief overview of research paradigms in education research. The focus of the study are three main paradigms: positivist, interpretivist, and critical theory with a brief review of the four components of each research paradigm - namely, ontology, epistemology, methodology, and methods.
Position of Quasi-Experimental Pedagogy - A Paradigmatic Debate
The choice of the philosophical basis and perspective when researching a certain phenomenon should be founded in the needs and requirements of the study, and it should not be insisted on only one philosophical point of view while excluding all others (Rehman and Alharthi, 2016). On the other hand, there are numerous paradigms that can guide us in the research process, thus relying on different paradigmatic schemes in order to conceptualize and classify research.
Pedagogy research is generally classified into two broad categories: qualitative and quantitative research. Each approach has its own methodology and terminology. Quantitative research uses measurement, whereby data is tested to examine pre-set hypotheses. It is a type of research which requires controlling variables. On the other hand, in qualitative research the focus is on research in the natural environment. No hypotheses are set here, but this type of research is distinctive because it can result in hypotheses and the creation of new theories. It is often thought that qualitative research is the precursor to quantitative research (Preglej, 2014). The terms quantitative and qualitative research generally mean more than different ways of gathering information and represent divergent assumptions about the nature and purpose of research in pedagogy (Bryman, 1988; Gojkov, Krulj and Kundacina, 1999; Guba and Lincoln, 1988; Halmi, 2005; Howe, 1988). Qualitative research describes events in their natural environment. Instead of designing experiments and controlling variables artificially, when conducting qualitative research, researchers use anthropological and ethnographic methods to study respondents (Lowhorn, 2007). Different research methods can be used in qualitative research. The qualitative method with qualitative analysis is one of the studied reality properties. We distinguish the following research methods within qualitative research: historical, comparative, genetic, functional method, structural method, system analysis, monographic method, case study method, dialectical method.
According to the quantitative approach, social phenomena should be treated in the same way natural phenomena are treated by scientists from the field of natural sciences. Quantitative research is a research methodology with a holistic approach that the researcher uses during the research process. The quantitative method deals with the quantification and analysis of variables in order to arrive at research results. It involves the use and analysis of numerical data to answer the following questions: who, how much, what, where, when and how? Quantitative research explains the problem by presenting data with numerical indicators, while the analysis and interpretation of research results is based on mathematical methods, especially statistics.
Figure 1. Non-experimental versus experimental research
Once the decision to conduct a quantitative study is made, it must be designed. There are two main types of quantitative research designs: experimental and non-experimental designs. Experimental pedagogical research deals with cause-and-effect relationships (also known as causal links and relationships) and strives determine the effects of pedagogical procedures in a systematic way (Knezevic-Floric and Ninkovic, 2012; Fajgelj, 2010). In experimental research, we manipulate variables in order to examine the influence of one variable on another, while in non-experimental research, variables are not manipulated.
NON-EXPERIMENTAL RESEARCH
EXPERIMENTAL RESEARCH
V
V
Quasi-Experimental Research in the Context of Experimental Research
The main reason for the rise of experimental pedagogy should be sought in the development of social and natural sciences, as well as in the circumstances of the 19th and the 20th century. Experimental research first emerged in natural sciences, and then spread to the field of social-humanistic sciences. Under the influence of the positivist research paradigm, there was a tendency to separate from deductivism and join natural sciences. The focus shifted from the field of psychology to the issues dealt with in pedagogy. Experimental pedagogy is empirical. The methodology dealt with in experimental pedagogy is exact and it is used to determine definite data, i.e., it looks for facts and evidence that can replace research hypotheses. A synonym for experimental pedagogy is "new pedagogy," although the term "scientific pedagogy" can also be found. In the first half of the 19th century, pedagogy spread its knowledge, developed the methodology of studying pedagogical problems and replaced theoretical and deductive studies of pedagogical phenomena with experimental, i.e., empirical research. Historically, in the period between 1880 and 1990, pedagogical laboratories that explored various pedagogical phenomena (teaching, teaching methodology, opinion and memory, curricula and programs, school systems, etc.) were opened (Nahod, 1999).
Experimental research is a systematic and scientific approach to research in which the researcher manipulates one or more variables, and controls and measures any changes in other variables. The goal of experimental research is predicting phenomena. In most cases, the experiment is designed in such a way so that some types of causal relations can be explained. Experimental research describes the process which the researcher undergoes to observe whether manipulating variables leads to certain results, i.e., whether the manipulation directly causes a certain outcome.
When research is carried out, there is no criterion that will immediately show that a certain design is better than the other, or that some experiment is more valuable than the other. Evaluating research quality is done through the methodological basis of the research, originality, novelty, and significance, and these are the so-called generic research evaluation criteria. Originality implies that the research is based on new data and on innovative approaches to researching complex pedagogical problems.
In the last few decades, a wider use of quasi-experimental research in social sciences has been observed. This trend, partially derived due to the "credibility revolution" in social sciences, is notable together with the increasing use of randomized controlled trials with the purpose of testing causal relations and conclusions (Gopalan, Rosinger and Ahn, 2020). Quasi-experimental research design should be brought as close as possible to the benefits of real experimental designs in a natural school environment.
The main difference between experimental and quasi-experimental research lies in the distribution of respondents into groups. In experimental research, distribution of respondents into groups is randomized to reduce bias. In quasi-experiments, such a random distribution is not possible. The control group serves as a comparison group. In order to retain the advantages of experimental research (environment control) as much as possible, it is crucial to ensure that the experimental and comparison (control) groups are as similar as possible. This is not easy to do because the number of variables that can affect outcomes is considerable. Therefore, the best thing to do is to think carefully about the factors that may affect the results. The control and experimental groups should be as similar as possible in the following sociodemographic characteristics: socioeconomic status, gender, ethnicity, ability, etc. The researcher must try to collect as much data as possible on as many variables as are believed to be relevant to the outcomes of the quasi-experimental research. Statistical control of the effects of these variables can then be attempted. it is obvious that from the point of view of establishing causality this is not an efficient method (Gopalan, Rosinger and Ahn, 2020; Muijs, 2004; Levy and Ellis, 2011).
Everything mentioned above makes it clear that it is necessary to find a group that matches the experimental group, and this is not an easy task at all; lack of randomization may lead to research bias. Quasi-experimental research designs have one obvious advantage over experimental designs, which is that they are conducted in a natural setting. If we discover the effects of a program, we can be sure that they occur in natural, not only laboratory, conditions. Such quasi-experimental research is suitable for evaluating some new initiatives and programs in the educational process (Muijs, 2004).
Quasi-experimental research designs test causal hypotheses. In both experimental and quasi-experimental research, a treatment, an intervention, or an experimental factor is used.
Groups of qualitative indicators of experimental and quasi-experimental research are as follows. Literature review is the key element for the conceptual design of the research; It is important that the review presents the existing information and grounded theory for the proposed research. Literature review should reflect recent and basic research studies in the field. If there is no literature to support a particular problem, the researcher should state this clearly. Regardless of whether the researcher is proposing
an innovative approach for which there is little existing empirical evidence or interventions, literature review should provide appropriate arguments for the importance of investigating a particular research problem; Highlighting the importance of re-investigating a certain issue; Presenting arguments supporting the intervention; Presenting sufficient arguments on why treatment is necessary for a certain group; Using adequate procedures to ensure that groups are comparable under all conditions; The intervention is clearly described; A faithfully described research procedure; A clearly described apparatus; Providing evidence for the validity and reliability of the obtained measures; Effects of intervention measures are recorded on time; The selected data analysis techniques correspond to the research questions and research hypotheses; The variability of the sample should be represented by adequate statistical parameters.
Real experiments and quasi-experiments have different goals. Real experiments are conducted in order to study a phenomenon, while quasi-experiments have a practical purpose. Quasi-experiments are aimed at discovering causal links and relationships based on correlations between the phenomena themselves. They are said to be "halfway between a passive observation and a true experiment" (Milas, 2005, p. 221).
To sum up, what guides researchers when choosing a research sample is debatable. If the experimental method is used, the researcher must randomly select the research sample. If they use the quasi-experimental method, they can use a deliberate and convenient sample. This can be done by researchers, students, teachers, reflective practitioners. From an epistemological and methodological point of view, it should be pointed out that if the researcher uses the experimental method with a preselected sample from the population, this is not a real experimental design, but a quasi-experimental study. For example, if a teacher introduces a new method in their class, this is not the experimental method, but the quasi-experimental method. In this sense, the aim is to establish the right terminology and perform quasi-experimental research from the shadows of experimental research.
The Mixed Method Reconciling Paradigms Through Quasi-Experimental Research
The relationship between the theoretical and the empirical approach to the study of the phenomenon or research problem is presented, together with the relationship between qualitative and quantitative methods in pedagogical research. As both types of research are beneficial (what can be learned by one method cannot be learned by the other due to different philosophical starting points to which they epistemologically belong), it cannot still be argued that one or the other perspective in research should be strictly followed. The solution is found in triangulation, that is, a mixed methodology that manages to reconcile these two paradigms through the methodological element of methods, techniques and research instruments that can be combined in empirical studies.
The mixed method actually refers to methods of data collection, data analysis and interpretation of evidence, that is, the analysis and interpretation of research results. The terms "mixing, mixed" are essential because this is a key step in a certain phase of the research process. "Purposeful data integration enables researchers to seek a more panoramic view of their research landscape, viewing phenomena from different viewpoints and through diverse research lenses" (Shorthen and Smith, 2017, p. 74-75). For example, in a randomized controlled trial, quantitative data can be collected to examine knowledge about a problem, while qualitative data is used to analyze certain experiences. This point of view is characteristic of both natural and social sciences. In experimental research, for example, the researcher may conduct an interview with respondents to examine the desired phenomenon, but they may also use a questionnaire or a rating scale to assess some characteristics of the investigated phenomenon. This is a typical example of combining a qualitative and a quantitative method because the interview is an instrument that belongs to qualitative research, while the assessment scale belongs the quantitative paradigmatic orientation.
The complexity of the educational process and the very nature of educational problems require the integration and combination of qualitative and quantitative methodology in research. The basic characteristics of these methodologies point to the need of using the mixed method. Quantitative methodology relies on large samples to analyze certain parts of the population, while qualitative methodology is based on small samples that are described and analyzed in a detailed manner. Combining them ensures the analysis of individual parts of the population, but also the understanding of their full complexity. Using the mixed method implies that the researcher is well acquainted with the basics of both qualitative and quantitative methodologies (Denzin and Lincoln, 2000).
It is necessary to point out that even in the context of experimental and quasi-experimental research, one should not stick to only one research tradition and that through the application of different research methods it is possible to reconcile the irreconcilable.
Concluding Remarks
In summary, each paradigm has its own advantages and disadvantages, that is, each paradigm has specific features and a unique role in the process of researching upbringing and education, especially in pedagogy. By using a multifaceted approach, not limiting ourselves to one paradigm, we can have a holistic framework and approach in investigating our research question. It is firmly believed that connecting paradigms in research ensures the quality, validity, reliability and relevance of the facts.
Furthermore, when the researcher is well acquainted with the philosophical, epistemological and methodological postulates, and paradigmatic frameworks, they can use this knowledge to change their views on the research methodology they are dealing with. For example, experimental research involves studying the effect of a systematic manipulation of one or more variables on another variable. To conduct a "true" experiment, researchers must select a randomized research sample. By random distribution, each respondent has an equal chance of entering the research process. Sometimes, researchers cannot randomly select respondents. Instead, the experimenter will often use already formed research groups, such as a class community, and conduct the research in the classroom. In this case, the research is described as quasi-experimental. In non-experimental quantitative research, the researcher identifies variables and may look for relationships between them but does not manipulate the variables. There is a tendency to assert the function and importance of research not only among researchers, but also among pedagogues, teachers, and students. It is important that researchers undertake the role of practitioner in the research process. Researchers should see the research problem as an action by which the practice is constantly changing and developing. As already mentioned, the realization of the research is described from the point of view of the subject's nature, research goal, sample, methods, techniques and instruments, as well as data processing. Each study is conducted with the aim of understanding the research problem better (Bandjur and Potkonjak, 1999; Hebib and Matovic, 2012). The relevant data can be collected in both qualitative and quantitative form or by constructively combining them.
It is important to point out to the scientific public that there are different types of quasi-experimental and experimental research designs. But in order for them to be further affirmed in special segments, it is necessary to define quasi-experimental research and understand the crucial difference between an experiment and a quasi-experiment. To achieve that goal, methodological, research and statistical literacy is necessary, especially for teachers who tend to improve themselves and the educational system. When the researcher is "qualified" in this domain, there is no doubt that they will conduct the research appropriately and write a proper report on its implementation, as well as distribute it to the scientific public. On the one hand, one of the goals of this paper was to gain a certain perspective of the research process, and on the other hand, it was to encourage greater argumentation of the practical findings of experimental research and affirmation of quasi-experimental research in pedagogy.
Conflict of interests
The authors declare no conflict of interest.
References
Al-Ababneh, M. M. (2020). Linking ontology, epistemology and research methodology. Science & Philosophy, 8(1), 75-91.
https://doi.org/10.23756/sp.v8i1.500 Antwi, S. K., & Hamza, K. (2015). Qualitative and quantitative research paradigms in business research: A philosophical reflection. European journal of business and management, 7(3), 217-225, Retrieved from https://www.iiste.org/ Journals/index.php/EJBM/article/view/19543 Bandur, V., & Potkonjak, N. (1999). Metodologija pedagogije [Methodology of pedagogy]. Beograd: SPDJ Bryman, A. (1988). Quantity and quality in social research. London: Routledg Craig, S. B., & Hannum, K. M. (2007). Experimental and quasi-experimental evaluations. The handbook of leadership development evaluation, 19-47. Retrieved from https://www.researchgate.net/publication/238733566_EXPERIMENTAL_AND_ QUASI-EXPERIMENTAL_EVALUATIONS Denzin, N. K., & Lincoln, Y. S. (2000). Introduction: The discipline andpractice of qualitative research. In N. K. Denzin & Y. S.
Lincoln (Eds.). Handbook of qualitative research (2nd ed., pp. 1-28). Thousand Oaks, CA: Sage. Fajgelj, S. (2010). Metode istrazivanja ponasanja [Behavioral research methods]. Beograd: CPP. Gojkov, G., Krulj, R., & Kundacina, M. (1999). Leksikon pedagoske metodologije. Vrsac: Visoka skola za obrazovanje vaspitaca. Gopalan, M., Rosinger, K., & Ahn, J. B. (2020). Use of Quasi-Experimental Research Designs in Education Research: Growth, Promise, and Challenges. Review of Research in Education, 1(44), 218-243. https://doi.org/10.3102/0091732X20903302 Guba, E. G., & Lincoln, Y. S. (1988). Naturalistic and rationalistic enquiry. Educational research, methodology and measurement: An international handbook, 81-85. https://www.researchgate.net/proflle/Russell_Gersten/publication/228355202_ Quality_Indicators_for_Group_Experimental_and_Quasi-Experimental_Research_in_Special_Education/
links/02e7e52c1eb8b7f8a2000000.pdf Halmi, A. (2005). Strategije kvalitativnih istrazivanja u primijenjenim drustvenim znanostima [Qualitative research strategies in applied social sciences]. Naklada Slap.
Hebib, E., & Matovic, N. (2012). Istrazivacki rad skolskog pedagoga [School pedagogues and research]. Nastava i vaspitanje, 61(1), 67-82. Retrieved from https://reff.f.bg.ac.rs/handle/123456789/1387 Howe, K. R. (1988). Against the quantitative-qualitative incompatibility thesis or dogmas die hard. Educational researcher,
17(8), 10-16. https://doi.org/10.3102/0013189X017008010 Kamal, S. S. L. B. A. (2019). Research paradigm and the philosophical foundations of a qualitative study. PEOPLE: International
Journal of Social Sciences, 4(3), 1386-1394. https://dx.doi.org/10.20319/pijss.2019.43.13861394 KneZevic-Foric, O., & Ninkovic, S. (20l2). Horizonti istrazivanja u obrazovanju [Horizons of research in education]. Novi Sad: Filozofski fakultet, odsek za pedagogiju. Retrieved from https://digitalna.ff.uns.ac.rs/sites/defaultfiles/db/books/ H0RIZ0NTI%20ISTRAZIVANJA%20U%200BRAZ0VANJU.pdf Levy, Y., & Ellis, T. J. (2011). A guide for novice researchers on experimental and quasi-experimental studies in information systems research. Interdisciplinary Journal of information, knowledge, and management, 6, 151-161. Retrieved from https://core.ac.uk/download/pdf/51072279.pdf Lowhorn, G. L. (2007, May). Qualitative and quantitative research: How to choose the best design. In Academic Business
World International Conference. Nashville, Tennessee. Retrieved from https://ssrn.com/abstract=2235986 Milas, G. (2005). Istrazivacke metode u psihologiji i drugim drustvenim znanostima [Research methods in psychology and
other social sciences]. Zagreb: Naklada Slap. Muijs, R. D. (2004). Doing quantitative research in Education with SPSS. New Delhi: Sage Publications, London, Thousand
Oaks. Retrieved from https://www.modares.ac.ir/uploads/En-Agr.Doc.AgriculturalExtension.18.pdf Nahod, S. (1999). Sto godina od uvodenja eksperimenta u pedagogiju [One hundred years since the introduction of experiments in pedagogy]. Nastava i vaspitanje, 5(48), 618-639. http://scindeks.ceon.rs/article.aspx?query=ISSID%26and%26911 &page=7&sort=8&stype=0&backurl=%2Fissue.aspx%3Fissue%3D911 Preglej, L. (2014). Istrazivanja u nastavi. Educatio biologiae: casopis edukacije biologije, (1), 100-116. Retrieved from https:// hrcak.srce.hr/148934
Rehman, A. A., & Alharthi, K. (2016). An introduction to research paradigms. International Journal of Educational Investigations,
3(8), 51-59. Retrieved from http://www.ijeionline.com/attachments/article/57/IJEI.Vol.3.No.8.05.pdf Shorten, A., & Smith, J. (2017). Mixed methods research: expanding the evidence base. Evidence-based nursing, 20(3), 74-75.
http://dx.doi.org/10.1136/eb-2017-102699 Sladoje Bosnjak, B. (2018a). Proces naucnog saznanja u pedagogiji [The process of scientific knowledge in pedagogy]. Istocno Sarajevo: Maticna biblioteka.
Sladoje Bosnjak, B. (2018b). Epistemologija pedagogije [Epistemology of pedagogy]. Godisnjak za pedagogiju, 3(1), 47-53. Sladoje Bosnjak, B. (2019). Teoantropoloski aspekti u vaspitanju: doprinos istoriji nacionalne pedagogije [Theoanthropological aspects in education: contribution to the history of national pedagogy]. Beograd: Jasen.
