CC BY
131
DIDACTIC TRENDS AND PERCEIVED TEACHERS' TRAINING NEEDS IN HIGHER EDUCATION: A CASE
STUDY
MA Mónica A. Bautista, Corporación Universitaria Minuto de Dios, Colombia E-mail: monica.bautista@uniminuto.edu Ed.D. Marisol E. Cipagauta, Corporación Universitaria Minuto de Dios, Colombia E-mail: mcipagauta@uniminuto.edu
A R T I C L E I N F O Case studies
Received: October, 23.2019. Revised: November, 28.2019. Accepted: December, 01.2019. doi: 10.5937/IJCRSEE1903071B
UDK
371.13(862) 37.026(862)
Keywords:
educational innovation, didactic trends, higher education, innovative practices, teachers training.
A B S T R A C T
Today's world requires people who manage their learning and professional updating processes, which claims for a change in educational practices in all levels of education. However, in Higher Education, this transformation relies mostly on teacher's efforts to innovate within the learning environments, requiring institutional efforts to generate strategies towards educational innovation. This paper presents the analysis of didactic trends and training needs of teachers at UNIMINUTO Colombia, based on the data gathered at the Annual Meetings for Innovative Pedagogical Practices and an additional survey applied on-line, to determine teachers' training needs and, furthermore, institutional strategies to strengthen the teaching-learning process. The study uses a mixed methodology through a concurrent triangulation design with qualitative and quantitative methods, with an exploratory scope; both quantitative and qualitative parts of the study counted on a voluntary sampling method. Trend analysis shows that the most used didactic methods are Project-Based learning, Research-Based Learning, and Collaborative Learning. However, the analysis of teachers' appropriation of knowledge about didactics, especially on those strategies, is low or basic, which compared to the training needs expressed by teachers demonstrate that training processes in educational innovation and new teaching methods is crucial to help educational innovation initiatives to evolve. The study stablishes a trend towards the use active-learning methodologies in pedagogical practices, highlighting the necessity of teachers' training in how and when to use them, and setting the importance of including communication skills as a topic in teachers' training programs.
© 2019 IJCRSEE. All rights reserved.
1. INTRODUCTION
Teachers' training programs in higher education focus on curricular activities, pedagogy, and techniques related to students' competencies achievement, but nowadays, areas such as emotion, leadership, and didactics gain great importance as mediators between teachers and their students. One of the primary purposes of this is to enhance professional and personal development in a self- management basis. Corresponding Author
Ed.D. Marisol E. Cipagauta, Corporación Universitaria Minuto de Dios, Colombia
E-mail: mcipagauta@uniminuto.edu @©@©
This work is licensed under a Creative Commons Attribution - NonCommercial - NoDerivs 4.0. The article is published with Open Access at www.ijcrsee.com
Active learning strategies have changed the roles in the learning process, stablishing the student as the active player of it on the side of the student himself; but at the same time, it challenges teachers in order to design interactive experiences that allow them to see how to apply knowledge in real situations.
On the other hand, we see how teachers already create many innovative experiences to give cutting- edge responses to a variety of students within the learning environment, each one with different perspectives, learning styles, preferences, and expectations on the higher education process. Meanwhile, higher education institutions keep thinking about creating training programs to strengthen the teaching activity and teachers' performance; but some of them are based on foreign models and other institutions' current programs rather than on the specific needs and possibilities they have.
At the Center for Teaching Excellence in UNIMINUTO, in charge of the design, implementation and evaluation of training programs for more than 4700 teachers in 64 different places all over the country, each one with different realities and resources, it has become essential to know what our professors actually do to improve their teaching practice, how they address learning objectives and what training programs must be created, based on identified needs and knowledge gaps.
Therefore, this study aims first to identify trends within the didactic techniques employed by higher education teachers among the institution, based on their current designs of innovative teaching practices. A second objective point towards the analysis of the degree of didactics' knowledge they currently possess, by analyzing the score given by jurors about the correspondence between the didactic strategy they refer and the practice shown in the methodology section of the texts, to determine possible training needs.
The third objective is to stablish training needs from teacher's perspective, to determine whether there is or not a correspondence between what the findings on the Annual Meeting and their requests, which allows the Center to create training programs from a more pertinent perspective.
1.1. Educational innovation and higher education
Educational innovation is specifically related to all those didactic and methodological strategies that are being incorporated in the educational processes, and that are looking for significant changes in order to respond to the needs of new population groups, new generations from a local and global perspective like decolonizing learning (Ferguson, et. al, 2019) and new teaching modalities such as virtual environments; which implies transformations in terms of resources for learning, roles of teachers and students, teaching and incorporation of tools such as information and communication technologies in pedagogical intermediations (Observatorio de Innovación Educativa del Tecnológico de Monterrey, 2017; Murillo, 2017).
The purpose of educational innovation is far from being merely instrumental and is to become a topic of high interest and impact at all educational levels. In higher education, it acquires great importance given the fact that is there where new professionals are
born by "actively understanding and building up meanings and skills" (Lasauskiene and Rauduvaite, 2015, p. 788) who will assume various roles and responsibilities in an increasingly interconnected and interdependent world. However, these connections must emerge from the creation of strategies that direct the efforts, and here is where we begin to find crucial differences between countries and cultures.
For example, the Department of Education, Universities and Research Administration of the Autonomous Community of El Pais Vasco (2008) issued a document givinig clear guidelines regarding the objectives and lines to which the efforts in educational innovation should be oriented. They stablished 3 fundamental ones:
1) innovation for inclusion (not only in terms of disability), 2) innovation for institutional improvement and 3) innovation for integration with the society of the 21st century. However, in Colombia, innovation is still considered as ICTs the integration of ICT (Information and Communication Technologies) into learning environments, and unfortunately, from the planning of these processes, other perspectives that also have to do with educational innovation such as the central axis of work in the country (Ministry of Education Republic of Colombia, 2016).
However, universities and Higher Education institutions have been able to approximate realities from other places, and this has generated the need to implement new strategies to expand the concept of educational innovation to meet the needs of professionals in training and the external sector and to stay current in an increasingly competitive market. It also led to understand that it is the teacher who materializes these bets and institutional ambitions within the different learning environments, and in the long run they are the ones who ensure the quality of education (Virtual Center for Education News, 2014). That also implies the need to generate strategies beyond pressure and control over teachers for compliance with regulations, which leads to rethinking teacher training and reorienting it towards continuous accompaniment processes for the transformation of teaching (Imbernon, 2011).
1.2. Teachers training research
Among the research developed in higher education around teachers, there are several related to teaching styles (Laudadio and Da Dalt, 2014), teaching competencies (Rivadeneria, 2017) the use of technologies in learning environments (Blink Learning, 2018; Ministry of Education Republic of Colombia, 2016), among other competencies that have allowed the construction of various instruments and indicators to gather information about it (European Union, 2017).
However, the materialization of the teacher competencies become real in the teaching practice, of which he is a student, he is the primary beneficiary, and some investigations try to determine the degree of effectiveness of the active methodologies, while others have focused on identifying trends in general (Hurtado, 2014). In this regard, Elena Hurtado, in 2014, conducts a review of research and concludes that it has not yet been possible to define which methodologies are most effective in specific fields, mainly due to the inconstancy of some authors regarding the publication of its pedagogical strategies in the same line, and the lack of contrast of results. Additionally, the author gives an account of the little academic production in this specific topic for Latin American countries to date, and after an updated review in databases such as EBSCO, the majority of longitudinal investigations, which would account for an impact In the long term in student learning, they remain mostly foreign (Roberts, 2018; McBride and Drake, 2016; Kilgo, Ezell Sheets, and Pascarella, 2015).
Specifically, on didactic trends in Higher Education, there was no scientific information available in databases, mainly because the studies are carried in a one by one basis to validate those practices and their achievements, instead of trends or institutional approaches towards educational innovations. Furthermore, this gap in knowledge also leads to a lack in research about the level of knowledge and appropriation that teachers have on didactic strategies. which for units in charge of teachers' training and qualification, this analysis becomes crucial in order to feed the planning process for a useful accompaniment.
1.3. Learning interests of teachers
As training programs are often planned by institutions based on their interests, and
teachers are not involved in the Professional Development planification process, there is also a gap with that topic. Louws, et. al. (2017) perfomed a study in secndary level teachers to determine what they wanted to learn within their professional development plan, and the results showed that they wanted to learn about ICT and subject-matter specific contents, mainly because of the continuous curricular changes taking place, which requieres teachers to be up to date in their subjects. However, some differences were found according to the career level of teachers, thus those with a long experience are more interested in climate and classrom management to adapt themselves to the current generations, while those teachers who are starting in their careers are more interested in classroom organization topics.
However, when cheking information regarding this topic in Higher Education levels, no information was retrieved, and the studies that cited the previously discussed research were drafted again towards teachers competencies, rather than questioning them about their professional interests and topics proposals.
This second gap in knowledge, and the current need of Teachers's Support Units in Higher Education Institutions, like the Center for Teaching Excellence aeiou, lead, to encourage efforts towards pertinent training programs, asking teachers directly becomes also a priority for the University and for this level of education as well.
1.4. Support Centers for Teachers
In Colombia and in other parts of the world, teaching support centers have emerged as an essential part of teacher qualification, many of which are gathered in the RedCrea (Crea, R. Red de Centros de Enseñanza-Aprendizaje: RedCrea, 2018): an organization dedicated to sharing professors' training strategies in different universities of the country since its creation in 2017, to which the aeiou Teaching Center of Excellence of UNIMINUTO (Aeiou Teaching Center of Excellence, 2018) created in 2016 adheres.
The commitment of UNIMINUTO towards educational innovation was strengthened and concretized with the creation of the Aeiou Center for Teaching Excellence, which aims primarily to generate training and accompaniment spaces for teachers in their process of teaching transformation. Before 2016, the teacher's exercise was not
very visible in the institution unless it made part within a specific investigation, which left a great void when trying to characterize the practices of teachers, their impact on student learning, and training needs towards educational innovation. Therefore, the Meeting of Innovative Pedagogical Practices has been the scenario that provided that window of observation and analysis.
A preliminary research was made with data from 2017 and 2018 calls of the Meeting, focusing mainly on the trending information, some of which will be used in the present research in order to make a context for the comparison with 2019 data (Moyano, M. E. C., and Pinzón, M. A. B., 2019). That firs study helped to organize training programs aimed at teachers' around the country using a webinar methodology, which according to the Center for Teaching Excellence aeiou, the coverage and access would be covered and the access to information would improve the results seen on the preliminary study.
When checking the stats for live participation and visualizations of the resources, obtained from Campus (the department supporting https://webinar. uniminuto.edu/) and YouTube, it showed that the average of live attendees in 2018 was 29 teachers, and the average of visualizations for the same year was 203 views, while teachers population at UNIMINUTO is above 4700, which is less than 10% of the total population.
The low participation and visualization rate sets the question whether teachers know that this platform and training resources and spaces do exist, or if there is another reason, why they do not engage with these resources, even though they say in the survey that those topics are actually, what they are interested. This also leads to stablish the question about the effectivity and convenience of the communication channels used to get in touch with them, taking into account that the main one used in 2018 was the company's email.
In 2018 and 2019, the promotional video inviting teachers to participate in the Meeting for Innovative Pedagogical Practices, had a specific invitation to review those resources, but in the end it's essential to identify if the message is really getting to teachers, if they are using those resources as information sources to prepare their pedagogical strategies and practices, or if they are not taking that into account. However, this specific topic requires a research for its own.
Based of the knowledge gaps found during the revision, and the interesst of Higher
Education support Centers such as aeiou, the present study aims to analyze: 1) the trends on didactic strategies use in innovative pedagogical practices gathering and comparing data from 2017, 2018 and 2019 2) determining new training needs from the analysis of the Pedagogical Practices Meeting extracted from jurors evaluation on the presented practices and 3) establishing training interests from teachers' perspective, contrasting it with the analysis of teachers' current knowledge and appropriaton of didactics, so the Center for Teaching excellence aeiou can establish a more pertinent and engaging training and accompanimenrt plan to promote better educational innovations.
2. MATERIALS AND METHODS 2.1. Design
The present study frames within a mixed methodology due to the use of quantitative and qualitative analysis, with a non-experimental approach and a concurrent triangulation design, but with an exploratory scope due to the lack of previous research on the specific institution (Hernández-Sampieri, Fernández, and Baptista, 2014).
2.2. Data collection and sample
There are two primary sources of information, the first one used mainly in a quantitative analysis of didactic strategies used by UNIMINUTO teachers during 2017, 2018, and 2019, using the Innovative practices presented to the second, third and fourth Meetings of Innovative Pedagogical Practices, hosted at UNIMINUTO. This analysis also focuses on analyzing the consistency score, given by the evaluators, that allow us to know whether the technique te teacher pleads to use is the one they use.
The second source is an open answer questionary, where teachers set up their perceived training needs, which led to quantitative and qualitative analysis. In the end, the triangulation process will take place on the teachers' perceived needs and the training needs resulting from the consistency analysis.
2.3. Innovative Pedagogical Practices and Evaluation
Data collection took place on the Annual Meeting of Innovative Practices, years 2017, 2018, and 2019. This event is open to every teacher at UNIMINUTO in Higher Education Level and focuses mainly on the presentation of those innovative practices that take place within learning environments, in order to aid and improve the teaching-learning process.
The necessary information the proposal must contain are: (a) pedagogical objective(s)' of the innovative practice, (b) the didactic strategy used or designed by the authors', (c) the methodology understood as the step by step of the implementation of the strategy, and (d) results, which are related to the pedagogical objectives directly.
An online form was the primary way to collect data from the authors, managed by Lime Survey, where information such as the branch they worked at, the disciplinary program, and the didactic strategy they used by selecting from a checklist, according to their knowledge and perception of it. The option "another" was also in the checklist for those participants who considered that any available option matched their strategy.
As the event advanced, from its 2017 to the 2019 version, some techniques were added later, which corresponded to the continuous updating process within the Center for Teaching Excellence aeiou and the information available for teachers via Webinars at Webinar. uniminuto.edu or during training sessions.
Figure 1 shows the didactic strategies from which authors could choose in 2017, and the additions in 2018 and 2019, from which they chose when making the registration process of their innovative pedagogical practice.
2017 Basic Strategies
»ProjectsBased Learning
• Research Based Learning •Problems Based Learning
* Service Learning
■ Challenges Based Learning ■Collaborative Learning
■ Learning Trough Simulation ■Adaptive Learning •Case Study
■Role Play Based Learning ■Another
Figure 1. Didactic strategies list for 2017 and additions for 2018 and 2019.
If the pedagogical practice did not match with any of the available categories, teachers had to select the option "Another", and for the 2019 call, if they had used more than one strategy, they had to mark the option "Multiple strategies".
2.4. Sample and sampling method
For 2017, the practices presented were 115, 143 for 2018, and in 2019 the call gathered 80 practices for a total amount of 338 texts; and since participation in the event is voluntary, the sampling, in this case, was non-random voluntary.
2.5. Evaluation of Innovative Pedagogical Practices
International and national jurors evaluated the innovative pedagogical practices texts by sending the results through an online form, where they graded the criteria explained in Table 1 to give a score on each category.
20JSAdditionaI Strategies
•Gaines Based Learning •Gamificalion •Flipped Classroom
2019 Additional Strategies
•QuestionsBased Learning •Cooperative Learning •Meaningful Learning •Ludificaiion •Design Thinking •Multiple strategies
Table 1. Core evaluation criteria
Criteria
Definition
The title must be authentic, creative and must reflect the core of the Pedagogical Practice
Title
The abstract is clear, concrete and allows identifying the core elements of the innovative pedagogical practice._
Abstract
Pedagogical The objective is clearly set and answers what is going to be developed in the
Qbjective(s) pedagogical practicesl how and why._
Theoretical The author supports his/her practices on theories and author referring the
base_didactic strategy and the discipline,_
, , . , - Shows ina systematic way the implementation on the innovative pedagogical _ practice, aiming to reach the pedagogical objective(s)._
Results show clearly if the pedagogical objective was achieved or not, and other
benefits from the practice._
Determines the coherence between the didactic strategy referred by the author
and the methodology_
Note: other criteria like the format of the text and sources where included on previous evaluation instruments, however in 2019 they were removed as the core elements within the practice narrative where stablished.
Results
Consistency
After the evaluation, scores for each criteria where added to obtain a final numerical score for each Innovative Pedagogical Practice, allowing to determine the winners on each event who presented their practices in the Annual Meeting.
A double-blind methodology was used to avoid possible biases within the evaluation process, and a specific section of comments was available for jurors to set additional comments for the authors, in order to help them improve their practices and texts for further meetings.
2.6. Trends and appropriation of didactic strategies by teachers
While the didactic strategy, referred by the author from the start, was the principal element for the analysis of trends in educational innovation, the critical criteria to analyze the knowledge that teachers have about didactics was the consistency score, aimed to grade the coherence between the didactic strategy referred by the author, and the methodology (didactic strategy) section. That served to evaluate the appropriation degree that teachers have on strategic methodologies.
• 0 to 1.99 Shallow appropriation or shallow knowledge of the selected strategies
• 2.00 to 2.99 Low appropriation or low knowledge of the selected strategies
• 3.00 to 3.59 Basic knowledge or appropriation of the selected strategies
• 3.60 to 4.19 Intermediate knowledge
or appropriation of the selected strategies
• 4.20 to 5 Advanced knowledge and appropriation of the selected strategies
The Center of Teaching Excellence use this scale mainly to have a categorization of knowledge degree on didactics to determine knowledge appropriation levels on didactics, and possible training needs on this specific topics.
2.7. What do teachers say they need? Requested services and topics of interest
The qualitative part of the data gathering took place in 2019, via an online survey with three open questions, of which two are part of the current research. Those questions were:
• "What services would you like the Center for Teaching Excellence to provide?"
• "What topics would you like the Center for Teaching Excellence to address or deepen, and why?"
These answers were assessed through inductive content analysis (Zhang and Wildemuth, 2017), where the categories emerged from the answers given by the teachers who participated in this survey (Hsieh & Shannon, 2005, in Zhang and Wildemuth, 2017). Those resulting categories were analyzed from a quantitative and a qualitative perspective.
In this phase, the sampling method was
also voluntary and gathered a total amount of 188 participants, excluding 19 of them due to lack of information or clarity in their responses.
3. RESULTS
As mentioned before, the main goals of the present research were:
1) To identify trends within the didactic techniques employed by higher education teachers among the institution, based on their current designs of innovative teaching practices.
2) To analyze of the degree of didactics' knowledge teachers possess through the categorization of the consistency between the didactic strategy referred by the teacher and the methodology section of the texts, to determine possible training needs.
3) To stablish training needs from teacher's perspective, to determine whether there is or not a correspondence between the findings on the Annual Meeting and their requests.
Results in this section address each objective in the specific order, starting with the trends identification, following with the consistency analysis and the identification of training needs, to end with the analysis of teacher's training needs and the triangulation of those needs with the findings for the second objective.
3.1. Didactic trends: 2017, 2018, and 2019
The information was collected on the first part on the call, by asking teachers what strategy or didactic they used for their practices
Based on the information regarding the didactic techniques that teachers claimed to have used in their innovative experiences, it was possible to classify and count the amount of innovative practices according to the selected strategy, shown in Figure 2. Which also lead to identify which were applied more often by teachers in 2017, 2018 and 2019: Projects Based Learning, Research-Based Learning, and Collaborative Learning, setting a trend for UNIMINUTO.
Didactic Strategies
Garriif iiation Design Thinking Cooperative Learning Non tpecrfieri AJ.i^t.i'ivt. | earring
Games Bawd Learning Role plav based learning Ludifieation Questions Based Learning Service Learning Problems Based Learning learning through Simulation Flipped Classroom Challenges Based Learning esse stvily Meaningful Learning Another Colaboretive Learning Research Based Learning Projects Based Learning Multiple strategies
■ 2019 ■ 2018 B 2Û1?
Figure 2. Amount of practices per didactic strategies. 2017, 2018, and 2019 at the Innovative Pedagogical Practices. extracted from Moyano, M. E. C. M. A. B., 2019.
pedagogical Comparison Meeting of Compilation , and Pinzón,
These results ratify the trend identified in the preliminary study (Moyano, M. E. C., and Pinzón, M. A. B., 2019) and also indicate a sustained interest from teachers in strengthening competencies, skills and abilities in students that are usually related to Project-Based Learning, such as responsibility, autonomy and self- dependence, problem solving, interdisciplinary approaches, communication and social abilities (Zhang and Wildemuth, 2017). While it also allows the student to link reality to what they are learning in the classroom, it increases motivation among students (Galena de la O, 2006). Moreover, it allows them to prepare for real-life challenges, increasing employability, and career development (Harvey et al. 2002; Kumar, 2007. in Leal Filho, Shiel, and Paco, 2016).
Collaborative work, which is the primary goal of collaborative learning, has been spotted as one of the critical competencies that new professionals must acquire for the 21st century within different careers and contexts (Pang, Wong, Leung, and Coombes, 2019; Tan, Yang, Koh, and Jonathan, 2016; Jogerst, et. al. 2015), in order to develop other skills and abilities that are required within the
occupation field like collaborative problem solving (Sun et. al., 2020). Alternatively, engaging with people at work (Fernandez-Araoz, 2014), for example, in coaching activities with peers, and generating willingly new proposals (Cumberland, Herd, Alagaraja, and Kerrick, 2016) that allow companies to renew their processes or gathering new ideas to remain current or stay on top among their competitors.
Last, but not least, research competencies as the principal goal of the Research-Based Learning, are necessary not just for academic jobs, but for each profession to keep knowledge updated, discover new paths and better ways to address current problems and future ones. Those are also related to critical and analytical thinking and data analysis, both important for employers (Pang, Wong, Leung, and Coombes, 2019) and other actors.
Other strategies, such as adaptive learning, simulations, and gamification, require a more significant technological development in order to get the desired pedagogical results.
The category another gathered a mix other strategies that were not available in the registration list, and occupied the second place in 2018, being however under the Projects Based Learning Strategy. Nevertheless, it also had the lowest number of practices under this classification in 2019, when the main category selected by teachers was multiple strategies in which teachers combined different techniques to create their pedagogical innovations, resulting in seventeen pedagogical proposals under this title (see Table 2). However, as those categories work with a variety of techniques and didactic strategies, they did not count in the present trend analysis, because it will require a further analysis on each proposal.
Table 2. Multiple Strategies Innovative Pedagogical Practices 2019
Practice ID
Didactic Strategy (strategies)
Meaningful Learning, Inidification
25
Project-Based Learning. Collaborative Learning, Sen-ice Learning
106
Games Based Learning, Problems Based Learning, Projects Based Learning
Challenges Based Learning. Cooperative Learning, Collaborative Learning, 164 Ludificatioru Corporal Expression and Motion, Creativity. Communication and Games
195
Meaningful Learning, Learning through Role Play
251
Cooperative Learning, Collaborative Learning, On-Site Learning, Role Play
264 Meaningful Learning, Case Study
306
Collaborative Learning. Learning by Simulation
321
Challenges Based Learning. Cooperative Learning
339
Challenges Based Learning, Case Study
340 Collaborative Learning. Meaningful Learning
341 Collaborative Learning, Learning by Simulation, Learning Through Role Play
355
Learning by Simulation, Learning Through Role Play
357 Research-Based Learning. Problems-Based Learning
373
Meaningful Learning, Design Thinking, Gamification
378
Projects-Based Learning. Gamification
398
Collaborative Learning, Case Study
Note: Information extracted from the application survey to the Meeting of Innovative Pedagogical Practices 2019.
This information gives the research new elements to think about: How do teachers put together these strategies? and How do they differentiate them among the process?. That also leads to a further look at how consistent are the pedagogical practices themselves with the strategies that authors are claiming to use.
3.2. Consistency between the claimed strategy and the pedagogical practice description, comparison 2017- 2018
To evaluate this item, researchers took the "consistency" punctuation given by evaluators to the practices, where they had to compare the "announced didactic strategy" against what they saw in the strategy's
description section.
Afterward, practices were grouped by didactic strategy to obtain the consistency score average, in order to determine the degree of appropriation and knowledge teachers showed on their proposals, about the didactic strategy they used.
A comparison between 2017 and 2018 show that in 2018, most didactic strategies obtained a lower level on consistency according to evaluators, compared to 2017 (see figure 3).
These results represent an apparent decreasing in knowledge about didactic strategies, despite the training processes that took place in 2018. However, it also may suggest a difference in the evaluation performed by jurors, which has to be revised furthermore. It is important to indicate that those strategies that do not have an associated score for 2017 appeared in 2018, which is why there is no data available to compare; and the categories Another Strategy, and Multiple Strategies are out of the analysis due to their particular features.
Consistency comparison 2017-2018
Gamiliralion Adaptative rearnpg Games taw : Learni ng Rolit pliybas«) (tirrins Learning PtoHems Based Learning Learning through Simularon Flipped Classroom CliJlletWs BJS«I lewnirti Ca« Sluíc Coliboralive learning Research Learni nç Project ta sed Leami ng
0 1 m ¿ 01A consistent1^ Average
i. a 4 s
fe 2 017 C ansí sí ency a ve rage
innovative strategies, according to the scale proposed in the methodology section of this document. Table 3 shows the obtained results: Table 3. Didactic strategy appropriation level, comparison 2017- 2018
2017 201S
Didactic Strategy appropria- appropria-
tion level tion level
Projects-Based Learning Basic Low
Research-Based Learning Basic Intermediate
Collaborative Learning Intermediate Basic
Case Study Basic Basic
Challenges Based Learning Intermediate Basic
Flipped Classroom Non applied Basic
Learning through Simulation Advanced Intermediate
Problems Based Interme- Interme-
Learning diate diate
Service Learning Intermediate Basic
Role play-based learning Intermediate Low
Games Based Learning Non applied Basic
Adaptative learning Advanced Basic
Gamiflcation Non applied Intermediate
Figure 3. Consistency scores between the claimed strategy and the strategy's description section, assigned by jurors. Compilation from Moyano, M. E. C., and Pinzón, M. A. B., 2019.
Taking the average score as a basis for the qualitative assessment about the appropriation and knowledge level that teachers have on
Note: Classification of didactics' appropriation level based on the Scale shown in methodology. Table resulting from the compilation of data shown in Moyano, M. E. C., and Pinzón, M. A. B., 2019.
According to data, teachers' level knowledge and appropriation level in 2017 mostly rated Intermediate, while in 2018, the predominant level was mostly basic, decreasing in almost every category, except for Research-Based Learning that went from Basic to Intermediate.
More interesting is the fact that two of the three trending didactic strategies went to a lower score, which leads to believe there is a possible lack of knowledge of teachers regarding didactic strategies specifications, or perhaps another weakness within the process
of creating or socializing these practices (Moyano, M. E. C., and Pinzón, M. A. B., 2019).
These results lead to new questions, additional to those placed of the preliminary research, that will take place on the discussion. The analysis for 2019 is still in progress, which is why those data set are not available in the current analysis.
3.3. What do teachers need? Requested services from teachers and topics of interest
Teachers' answers to the question: Which services would you need from the Center for Teaching Excellence? Produced nine categories, shown in figure 4. The most requested services are training programs and
personalized advisory.
Required Services 1 83j
Q Meetings 1 12
■fj On site advisory 1 5
' ) Pedagogic retirements 1 3
Q Personalized advisory 1 15
{Jj Resources development 1 8
( } Short Course? 1 3
| "j Teams conformation 1 4
Q Training programs 1 32
Q University exchange programs 1 1
Figure 4. Services requested to the Center of Teaching Excellence by teachers at UNIMINUTO. Image extracted from NVIVO.
Training programs, according to teachers, should tackle topics such as Learning Assessment, Innovative Practices Design, the use of Current and Innovative Tools in the Learning Environment, Pedagogy, and Didactic Strategies for Virtual Education, Content Development and Production, and Research within the Learning Scenario.
While they request the personalized advisory for:
"...la incorporación de temas como innovaciones pedagógicas a las investigaciones [the incorporation of pedagogical innovations to researches]" (Participant 62).
". problemáticas emergentes que planteamos los docentes [Emerging problematics set by teachers]" (Participant 142).
". innovación, proyectos de emprendimiento y trabajos de grado [innovation, entrepreneurship projects and
thesis] (Participant 57).
About topics of interest in which teachers want to deepen or explore, six mayor categories were identified (see Figure 5).
Q Q Tope is of interest 1 203
(~) Disciplinary topics 1 24
(J Other areas 1 17
(''. Personal development I 9
Q Research 1 24
> ( ; Strengthening of teaching 1 127
Q Students' «compsniment 1 2
Figure 5. Coding of topics of interest to teachers to be included in their training and qualification process. Image extracted from NVIVO.
The category with more references was Teaching Strengthening, which is the target of the present research, followed by Research and Disciplinary related topics. Within the first one, some subcategories were set: (a) Pedagogy and didactics, (b) ICT tools, (c) Academic publication, (d) Inclusive education, (e) Learning evaluation, (f) teaching for specific contexts and (g) collaborative work among teachers (see Figure 6).
Inclusive education showed two big topics: "Educational inclusion for cultural diversity and disability" (participant 41), especially related to "inclusive didactic strategies for inclusive education" (participant 126).
Figure 6. Subcategories among Teaching Strengthening. Image extracted from NVIVO.
While Academic Publication was related to topics such as "workshops on how to write
academic papers and research articles to participate in research calls and other events" (Participant 48).
About learning assessment, teachers ask for training in Assessment Methodologies (Participants 3, 23 and 119), Innovative Methods for Learning Assessment (Participants 81 and 119).
ICT Tools Use addresses specific training needs on how to use current and new tools within the learning environment, such as Moodle 3.5 suite, simulators that are already in the curriculum such as Sniffy, PsySim, MathLab, SIMDEF, MarketStrat, SIMPRO, MARKLOG, BioTK, SICIEM... among others included in the curriculum. Mainly because:
"La Universidad cuenta con muchas herramientas tecnológicas que permiten el desarrollo y la formación tanto de los estudiantes como de los formadores, pero en muchas ocasiones, los tutores no conocemos las herramientas y aplicaciones de todo con lo que la Universidad cuenta. [The University has many technological tools that allow the development and training of both teachers and students, but very often, we do not know about those tools and apps the University has]" (Participant 152).
Other topics in this category are apps to improve interaction processes (Participant 152), Virtual Learning Environments, Technology and Didactics (Participant 126), educational Apps and web tools (Participants 119 and 147), augmented reality (participant 8) and software development for specific topics and courses (Participant 98).
The interest in ICT training relies on the belief that by using ICTs ".hacen que el proceso de enseñanza aprendizaje sea más dinámico [makes the teaching-learning process more dynamic]" (Participant 40).
3.4. Pedagogy and Didactics as a concern for Higher Education Teachers
This category is one of the most important, where teachers let us know what they feel they need to learn regarding didactics and pedagogy, which are the main elements of those innovative pedagogical practices they propose. Some of the topics requested for training were Pedagogical Models and Methodologies (Participant 75), such as constructivism and critical thinking (Participant 32), Hermeneutics, Participatory
Action Research and Experiential Education (Participant 33), Ludification and virtual games to work with Game-Based Learning (Participants 136, 19, 44, 47), and methodologies for Virtual Learning Environments (Participant 131). As well as Transmedia for education (Participant 35), and Innovation and Creativity (Participants 45 and 57).
There are also other suggestions like How to apply pedagogy in companies and the business sector through Communities of Practice (Participant 116), and how to articulate pedagogical processes with learning communities and other relevant networks (participant 111).
However, the most requested services were workshops, experiential activities, and personal advisory on how to create didactic strategies from a critical and reflexive perspective, related to different areas of knowledge that also integrate technologies and other innovations and allow collaboration between students.
Therefore, they also ask to know about "... aula pedagógica o ejemplos exitosos en otras universidades a nivel nacional o internacional [... pedagogy in the classroom, and successful examples in national or international universities]" (Participant 92), as well as information regarding Higher Education Trends in teaching methods and practices and "... formación en la era digital [education in the Digital Age]" (Participant 124).
The main reasons behind education innovation and their demands are related to the teachers' interest on making learning more dynamic (Participants 41), meaningful and experiential for the students (Participant 89); more pertinent to each discipline and the curricular needs (Participants 66, 90, 99), complementing the teaching practice through complementary strategies and tools (Participant 95).
4. DISCUSSIONS
The analysis drawn from the results above and the previous study (Moyano, M. E. C., and Pinzón, M. A. B., 2019), is presented in three levels: the first one is related to the trends observed in the Meetings of Innovative Pedagogical Practices, which is a first effort towards the beginning of research in this particular field due to the lack of previous knowledge in it. The second one addresses
the teachers' knowledge and training needs, contrasted with the research by Louws, et. al. (2017).
4.1. Didactic trends and challenges:
first results on the matter
To start researching about didactic trends, the previous findings (Moyano, M. E. C., and Pinzón, M. A. B., 2019) specified a possible trend confirmed with 2019 results, stablishing Project-based Learning, Research-Based Learning, and Collaborative Learning again as the most used strategies by teachers in their innovative practices, drawn from the individual strategies analysis. Therefore, in this case, it is necessary to keep evaluating the following years to verify these results and expand the research to ask teachers about their motivation to select those specific didactic approaches.
4.2. Teachers' specific knowledge of didactic strategies for pedagogical
strategies design
In 2019, the Multiple Strategies category was the most selected by teachers. However, when checking the options up close, it could be seen that in some cases the selected didactics are actually embedded, as is the case of Projects-Based Learning or Problems-Based Learning, and Collaborative Learning or Cooperative Learning, taking into account that the first ones aim to improve collaboration and/or cooperation among students (Walker, Leary, Hmelo-Silver, and Ertmer, 2015; Lee, Huh, and Reigeluth, 2015). That leads to question whether the practices have specific phases to target collaboration or cooperation before the Projects-Based Learning or Problems-Based Learning phase takes place, or if teachers do not know that those are implied on those strategies.
Other cases as the practice 164 (See table 1) show a significant amount of strategies used in the pedagogical practice, but the selection itself shows some inconsistencies (collaborative and cooperative learning in the same practice, for example) and the use of communication means (corporal expression) and goals (such as creativity) in the didactic strategy section. That leads to more substantial questions regarding the actual knowledge that Higher Education teachers have on didactic strategies and learning goals when designing
activities and pedagogical practices.
The previous findings concur with the consistency results shown above, where the appropriation level of knowledge in didactic strategies is mostly Basic, and correlates with teachers' request on training programs addressing didactic strategies and didactics design; showing that some teachers already acknowledge their lacks, and the Center for Teaching Excellence aeiou already has training programs, especially virtual ones at webinar.uniminuto.edu.
4.3. What to reinforce in teachers' training?
As the preliminary study stated, it is crucial to identify if the consistency issue on the formulation of pedagogical practices depends on the lack of specific knowledge about didactic strategies and their specific elements, or if there is a misunderstanding related to the text itself and the way teachers do present their practices and results.
If the first, training programs must focus on a workshop basis where teachers can learn the specifics of different strategies ending in the implementation of those in their current courses, what requires a strong accompaniment in the design, implementation, and evaluation process of new didactic approaches.
If the second, training programs must be aimed to improve communication skills in teachers, both written and verbal, to enhance the narrative process and make it more accurate to tell what they do in the learning environments. That also leads to the discussion of whether the outcomes, resources, and publications resulting from these innovative efforts should be published for open access and how to do it properly.
Either possibility is supported on the current topics teachers are requesting for the next qualification cycles, but identifying which one is the most critical means it is necessary to read carefully into every paper to define which the most urgent need is, and that is the topic for further investigation.
However, as educational institutions know, to offer training programs is one thing, but to engage teachers in it is a different matter, which is why teachers' engagement has to be seen in the eyes of its beneficiaries.
4.4. Teachers' engagement in training as a challenge
This last part contrasts the quantitative findings about teachers' training need from the analysis, with the results found in the qualitative analysis of their responses when asked about their training interests and the research by Louws, et. al. (2017).
The first finding was that Higher Education Teachers are interested in having a training aimed at improving pedagogical competencies and habilities in contrast to what Louws, et. al. (2017) found. That could be explained because while secondary education requests necessarily from tachers to have strong pedagogical skills and training, Higher education focuses mainly on the professional training, therefore, teachers are often more versed in their disciplines than in pedagogy itself. However, specific training to be up to date in knowledge is alro requested bu Higher Education teachers, what could be more related to the discipline itself than to curricular changes only.
Findings from the analysis also show that teachers need to reinforce their knowledge to achieve educational innovations, but it is important to provide different options and sceneries where they can practice and apply their knowledge, rather than just getting theinformation. That sets up a challenge for accompaniment strategies, even more in a context with such variety as UNIMINUTO's. And also raise questions towards the creation of collaborative networks to allow teachers to share knowledge, strategies and to develop communities of practice, both in pedagogical matters and disciplinary ones.
Engagement, however, is only developed after the sense of belonging and being listened to arises; so the training planing must have into account teachers' requests in order to gather them around their interests, increasing participation and lowering dessertions, which in this case requieres a concertation with other areas among the university to generate a more complete startegy that does not depend only in teachers' interests, but shows them support in their efforts so educational innovations keep evolving.
5. CONCLUSIONS
Current strategies used by higher education teachers in UNIMINUTO want to encourage active learning methodologies
in the classroom, to activate students and change roles in the teaching-learning process. Especially, those aimed at the creation of projects and the strengthening of competencies such as teamwork, data analysis and self-regulation, are preferred by teachers in their innovative exercises. In spite of this, the analysis highlighted the need of going deeper and further in the evaluation of the texts presented for the Annual Meetings of Innovative Pedagogical Practices, to identify if the low coherence scores in the practices are only related to the lack of knowledge and appropriation of didactic strategies, or if there is a necessity to work on improving communication skills in teachers. Mainly because there could be an assumption of expertise on that specific matter, which is why a text review and a qualitative analysis of them is crucial for the following phases.
That will also help the Center for Teaching Excellence aeiou to establish the most urgent topics teachers need to improve their didactic designs or the narrative they use to share them, thus having that information, the training offer will be more accurate a to accompany teachers on their qualification path. Doing so requires reviewing the contact and communication strategies that the Center has to get in touch with them, and evaluate the points that would help to engage with them and make the training more effective.
In June 2019, The Center for Teaching Excellence developed a new platform through Yammer to create a network between teachers at UNIMINUTO, making that a new communication-interaction channel with teachers, where it is expected to see ideas exchange between them, participation and comments. So far, it has allowed communicating with teachers in another scenery, where they can comment and share their own interests, but is still in the starting phase.
The main conclusion extracted from this research is that, even though there are elements to improve, there is an intention from teachers to promote active learning in students and the acquisition of other abilities and competencies beyond knowledge procurement and concepts' memorization. Nevertheless, it needs institutional support through the creation of training and professional development processes that consider teachers' opinions and interests in order to be more engaging and, therefore, effective to achieve a real and sustained effort towards educational innovation.
This research will show a more robust and complete analysis in 2023, gathering information from 2017-2022, including those instruments and elements that have not been taken into account so far.
ACKNOWLEDGEMENTS
This article is a result of the project Educational Innovation at the Center for Teaching Excellence aeiou at UNIMINUTO, and it is based on all those teachers who have participated every year in the Annual Meeting for Innovative Practices, without whom this research could not be possible.
Conflict of interests
The authors declare no conflict of interest.
REFERENCES
Administración de la Comunidad Autónoma del País Vasco Departamento de Educación, Universidades e Investigación. (2008). Líneas Prioritarias de Innovación Educativa 20072010. [Priority Lines of Educational Innovation 2007-2010]. RGM S.A. Retrieved from: https:// www.euskadi.eus/contenidos/informacion/ dig_publicaciones_innovacion/es_sist_edu/ adjuntos/13_hezkuntza_sistema_000/000003c_ Pub_EJ_lineas_0710_c.pdf Blink Learning. (November 2018). IV estudio sobre el uso de la tecnología en la educación. [4th study about the use of technology in education] Retrieved from: https://www. realinfluencers.es/2018/11/13/iv-estudio-sobre-el-uso-de-la-tecnologia-en-la-educacion/ Centro de Excelencia Docente aeiou. (September 14, 2018). Centro de Excelencia Docente aeiou Web Site. [Center for Teaching Excellence aeiou Web Site] Retrieved from: http://www.uniminuto.edu/web/centroaeiou Centro de Formación e Innovación Educativa. (2016). Modelo de Innovación Educativa. Estrategias y acciones para generar la innovación educativa. [Educational Innovation Model. Strategies and actions to generate educational innovation] Retrieved from: https:// innovacionenredenlaencb.files.wordpress. com/2016/12/modeloinnovacioneducativa.pdf Centro Virtual de Noticias de la Educación. (May 5, 2014). La calidad en la educación depende directamente de la calidad de los profesores, los educadores y los currículos, Clare Kosnik. Retrieved from: http://www.mineducacion. gov.co/cvn/1665/w3-article-340967.html Crea, R. Red de Centros de Enseñanza-Aprendizaje: RedCrea (2018). [Network of Teaching-Learning Centers: RedCrea]. Retrieved from: http://redcrea.co.
Cumberland, D. M., Herd, A., Alagaraja, M., & Kerrick, S. A. (2016). Assessment and development of global leadership competencies in the
workplace: A review of literature. Advances in Developing Human Resources, 18(3), 301-317. https://doi.org/10J 177/1523422316645883 European Union [Unión Europea]. (2017). Marco Europeo para la competencia digital del profesorado [European Framework for Teachers' Digital Competency]. Retrieved from https://ec.europa.eu/jrc/sites/jrcsh/files/ digcompedu_leaflet_es-nov2017pdf.pdf Ferguson, R., Coughlan, T., Egelandsdal, K., Gaved, M., Herodotou, C., Hillaire, G., ... & Misiejuk, K. (2019). Innovating Pedagogy 2019: Open University Innovation Report 7. Retrieved from: http://oro.open.ac.uk/59132/ Fernández-Aráoz, C. (2014). 21st-Century talent spotting. Harvard Business Review, 92(6), 46-54. Retrieved from: https://hbr. org/2014/06/21st-century-talent-spotting Galeana De La O, L. (2006). Aprendizaje basado en proyectos. Investigación en Educación a distancia. Revista digital. Rescatado el, 16. [Project Based Learning]. Retrieved from: http:// ceupromed.ucol.mx/revista/PdfArt/1/27.pdf Harvey, L., Locke, W., Morey, A. (2002) Enhancing Employability, Recognising Diversity: Making links between higher education and the world of work. London: Universities UK and the Careers Services Unit. Retrieved from https://www.qualityresearchinternational. com/Harvey %20 papers/Harvey, %20 Locke%20and%20Morey%202002%20 Enhancing % 20employability.pdf Hernández- Sampieri, R., fernández, C., & Baptista, P. (2014). Metodología de la Investigación [Research Methodology] (Sixth ed.). México D.F.: McGraw Hill. Hurtado, G. E. (2014). ¿Cuáles son las tendencias en las metodologías de enseñanza de la última década en iberoamérica?-¿ Which are the trends in the teaching methods of the past decade in iberoamérica?. [Which are the trends in teaching methodologies for the last Decade in Ibero America?] Revista científica, 1(18), 8699. https://doi.org/10.14483/23448350.5564 Imbernón, F. (2001). Claves para una nueva formación del profesorado. [Keys to a new teacher training]. Revista Investigación en la Escuela, 43, 57-66. Retrieved from: http://hdl.handle.net/11441/60307
Jogerst, K., Callender, B., Adams, V., Evert, J., Fields, E., Hall, T., ... & Simon, L. (2015). Identifying interprofessional global health competencies for 21st-century health professionals. Annals of Global Health, 81(2), 239-247. https://doi.org/10.1016Zj.aogh.2015.03.006 Kilgo, C. A., Sheets, J. K. E., & Pascarella, E. T. (2015). The link between high-impact practices and student learning: Some longitudinal evidence. Higher Education, 69(4), 509-525. https://doi.org/10.1007/s10734-014-9788-z Lasauskiene, J., & Rauduvaite, A. (2015). Project-Based Learning at University: Teaching Experiences of Lecturers. Procedia - Social and Behavioral Sciences, 197, 788- 792. https://doi.org/10.1016/j.sbspro.2015.07.182 Laudadío, M. J., & Da Dalt, E. (2014). Estudio de los estilos de enseñanza y estilos de aprendizaje en la universidad. [Study of teaching styles and learning styles in college] Educación y educadores, 17(3), 483-498.
https://doi.org/10.5294/edu.2014.17.3.5 Leal Filho, W., Shiel, C., & Paco, A. (2016). Implementing and operationalising integrative approaches to sustainability in higher education: the role of project-oriented learning. Journal of cleaner Production, 133, 126-135. https:// doi.org/10.1016/j.jclepro.2016.05.079 Lee, D., Huh, Y., & Reigeluth, C. M. (2015). Collaboration, intragroup conflict, and social skills in project-based learning. Instructional Science, 43(5), 561-590. https://doi.org/10.1007/s11251-015-9348-7 Louws, M. L., Meirink, J. A., van Veen, K., & van Driel, J. H. (2017). Teachers' self-directed learning and teaching experience: What, how, and why teachers want to learn. Teaching and teacher education, 66, 171-183. https://doi.org/10.1016Zj.tate.2017.04.004 McBride, J. M., & Drake, R. L. (2016). Longitudinal cohort study on medical student retention of anatomical knowledge in an integrated problem-based learning curriculum. Medical teacher, 38(12), 1209-1213. https://doi. org/10.1080/0142159X.2016.1210113 Ministerio de Educación Nacional República de Colombia. (2016). La innovación educativa en Colombia. Buenas prácticas para la innovación y las TIC en Educación. Retrieved from: https://n9.cl/6cig
Moyano, M. E. C., & Pinzón, M. A. B. (2019). Las prácticas pedagógicas un reconocimiento a la innovación/Práticas pedagógicas reconhecimento á inovagao. [Pedagogical practices, an acknowledgement to innovation]. Brazilian Journal of Development, 5(9), 1481414825. https://doi.org/10.34117/BJDV5N9-081 Murillo, A. (2017). ¿Qué es innovación educativa. [What is ducational Innovation?] Retrieved from: https://observatorio. itesm.mx/edu-news/innovacion-educativa Observatorio de Innovación Educativa del Tecnológico de Monterrey. (October 2014). Reporte Edu Trends: Aprendizaje invertido. [Edu Trends Report: Flipped Learning] Retrieved from: http://www.sitios.itesm. mx/webtools/Zs2Ps/roie/octubre14.pdf Observatorio de Innovación Educativa del Tecnológico de Monterrey. (2017). Radar de Innovación Educativa 2017. Tecnológico de Monterrey. [Educational INnovation Radar 2017. Tecnológico de Monterrey] Retrieved from: https://observatorio.itesm. mx/radar-de-innovacin-educativa-2017 Pang, E., Wong, M., Leung, C. H., & Coombes, J. (2019). Competencies for fresh graduates' success at work: Perspectives of employers. Industry and Higher Education, 33(1), 55-65. https://doi.org/10.1177/0950422218792333 Representación de la UNESCO de Perú. (2016). Innovación Educativa. Herramientas de apoyo para el trabajo docente. Lima: CARTOLAN E.I.R.L. [Educational Innovation Support tools for teaching] Rivadeneria, E. (2017). Competencias didácticas-pedagógicas del docente, en la transformación del estudiante universitario. [Teaching-pedagogical skills of the teacher, in the transformation of the university student]. Orbis. Revista Científica de Ciencias Humanas, 13(37), 41-55. Retrieved from: http://www.
redalyc.org/articulo.oa?id=70952383003 Roberts, D. (2018). Active Learning Precursors in Multidisciplinary Large Lectures: A Longitudinal Trial on the Effect of Imagery in Higher Education Lectures. College Teaching, 66(4), 199-210. https://doi. org/10.1080/87567555.2018.1486802 Sun, C., Shute, V. J., Stewart, A., Yonehiro, J., Duran, N., & D'Mello, S. (2020). Towards a generalized competency model of collaborative problem solving. Computers & Education, 143, 103672. https://doi.org/10.1016/j.compedu.2019.103672 Tan, J. P. L., Yang, S., Koh, E., & Jonathan, C. (2016, April). Fostering 21st century literacies through a collaborative critical reading and learning analytics environment: user-perceived benefits and problematics. In Proceedings of the sixth international conference on learning analytics & knowledge (pp. 430-434). ACM. https://doi.org/10.1145/2883851.2883965 Unión Europea. (2017). Marco Europeo para la competencia digital del profesorado (DigCompEdu).
Retrieved from: https://n9.cl/2rdw Universidad del Norte. (17 de julio de 2017). Centros de apoyo a la docencia del país y el continente compartieron sus experiencias. [Teaching support centers in the country and the continent shared their experiences] Retrieved from: https://n9.cl/rihav Virtual Center for Education News [Centro Virtual de Noticias de la Educación] (May 5, 2014). La calidad en la educación depende directamente de la calidad de los profesores, los educadores y los currículos, Clare Kosnik [The quality of education depends directly on the quality of teachers, educators and curricula, Clare Kosnik]. Retrieved from http://www.mineducacion. gov.co/cvn/1665/w3-article-340967.html Walker, A. E., Leary, H., Hmelo-Silver, C. E., & Ertmer, P. A. (Eds.). (2015). Essential readings in problem-based learning. Purdue University Press. Retrieved from: https://books.google. rs/books?hl=en&lr=&id=KhF-BgAAQBA J&oi=fnd&pg=PR1&dq=Walker,+A.,+Lea ry,+H.,+Hmelo-+Silver,+C.,+%26+Ertmer, +P.+(Edits.).+(2015).+Essential+Readings+ in+Problem-based+Learning.+West+Lafaye te:+Purdue+University+Press.&ots=awo8o Uhv_p&sig=0YQf0MDIQP3uCPPNlornPp SV3to&redir_esc=y#v=onepage&q&f=false Zhang, I., & Wildemuth, B. (2017). Qualitative Analysis of Content. En B. Wildemuth (Ed.), Applications of Social Research Methods to Questions in Information and Library Science (Second ed.). Santa Bárbara, CA; Denver, CO: Libraries Unlimited.
