CC BY
3
RUSSIAN LAW JOURNAL Volume XI (2023) Issue 3s
EVALUATION OF THE RESULTS OF THE VIRTUAL LEARNING OBJECT FOR IMPROVING WATER MANAGEMENT AND CONSUMPTION HABITS IN THE INHABITANTS OF THE UPPER BOGOTÁ RIVER BASIN
MARIO FERNANDO CASTRO1, EDGAR CAMILO DÍAZ2, JIMMY ENRIQUE GARZÓN3, JAIRO AUGUSTO CORTES4, FREDYS A. SIMANCA H.5, JAIME ALBERTO PAEZ6
1 Profesor Investigador, Programa Ingeniería de Sistemas. Universidad Cooperativa de Colombia, Bogotá - Colombia. Email: mario.castro@campusucc.edu.co
2 Profesor Investigador, Programa Ingeniería de Sistemas. Universidad Cooperativa de Colombia, Bogotá - Colombia. Email: edgar.diaz@campusucc.edu.co
3 Profesor Investigador, Programa Ingeniería de Sistemas. Universidad Cooperativa de Colombia, Bogotá - Colombia. Email: jimmy.garzon@campusucc.edu.co
4 Profesor Investigador, Programa Ingeniería de Sistemas. Universidad Cooperativa de Colombia, Bogotá - Colombia. Email: jairo.mendez@campusucc.edu.co
5 Profesor Investigador, Programa Ingeniería de Sistemas. Universidad Cooperativa de Colombia, Bogotá - Colombia. Email: fredys.simanca@campusucc.edu.co
6 Profesor Investigador, Programa Ingeniería de Sistemas. Universidad Cooperativa de Colombia, Bogotá - Colombia. Email: jaime.paez@campusucc.edu.co
Abstract: The purpose of this article is to describe the results of the evaluation of a virtual learning object to help improve the consumption habits and management of water resources in the inhabitants of the upper reaches of the Bogotá River. The evaluation was developed in 4 stages.Stages 1 and 2 evaluated 20 users with the profile of students of the systems engineering program at the Universidad Cooperativa de Colombia.Stage 3 evaluated 10 users with advanced knowledge of the problems and the management of at least one computer system in the upper basin of the Bogotá river. Stage 4 evaluated 20 users inhabitants of the upper basin of the Bogotá river. The surveys of stages 1, 2, 3 and 4 were divided into 5 dimensions distributed by groups of questions that evaluate the perception of the developer and the users. This information generated positive results allowing to visualize the scope of the functionality and the perception of the users, additionally evaluated the impact of the application on the different types of users for the application of these systems in the upper Bogotá river basin, allowing generating important conclusions that establish the evaluation of the virtual educational environments as important supports in teaching, not only of environmental systems, and that should be evaluated by the users that integrally interact with the system. The above process allows for establishing essential conclusions resulting from the evaluation of virtual educational environments as necessary supports in the teaching process, not only of environmental systems; it establishes clear parameters for measuring virtual learning objects in which different professionals are involved in the development of educational environments and people in the learning process.
Keywords: Virtual Learning Object; MACOBA, Learning Management System, methodology.
Table of Contents
1. INTRODUCTION
2. METHODS
3. RESULTS
4. DISCUSSION
5. CONCLUSIONS
INTRODUCTION
1
A Virtual Learning Object [VLO] is a digital resource used in different educational contexts (Salvador et al., 2017). This Virtual Learning Object is used simultaneouslyto complement teaching processes in which the pedagogical area, the teaching design and the technological area intervene as actors in its elaboration (Cóndor-Herrera et al., 2021).
Among the main advantages that Virtual Learning Objects have it is highlighted use as an extension that manages to know the student's progress, the generation of evaluative content processes of knowledge and recognition of information and adaptability to any LMS platform (Learning Management System) or in virtual platforms (Nohora & Moreno García, 2017). Among the resources that can be implemented are theories, explanations, didactic resources, activities, practice exercises and evaluations to facilitate the use through any digital equipment that facilitates the study and understanding of organized topics, such as, for example, in programmatic contents of institutional subjects (Barari et al., 2020). The World Wildlife Fund [WWF] defines river basins as "the exact point where the waters of the upper reaches meet and where the main river begins to have a defined course."Due to the high exploitation of natural resources in Colombia (Muratoglu, 2019), it is necessary to make people aware of the efficient management of resources that, in addition to generating a great negative impact on the surrounding ecosystem due to misuse, greatly reduce the sustainable development of the country (Burgos-Ayala et al., 2020), which is why, because the Bogotá river basin has a significant value in the social and economic development of Colombia (Sanabria-Suarez et al., 2017), the national government has outlined goals that allow the populations inhabiting the account to access education in the efficient and sustainable management of the resources that exist in the area, and in particular, that of water (Ramírez & Santana, 2019). Another problem faced by the population living in the area at the socio-environmental level is the inadequate management of waste (Sánchez-Alfonso et al., 2020) due to the inefficient management of water resources in daily tasks, the invasion of populations in the watershed's watershed, deforestation and lack of knowledge of water management (Díaz-Casallas et al., 2019).
It is important to know the causes and consequences of the lack of knowledge of the negative environmental impact of the inadequate processes carried out in the upper basin of the Bogotá River (Nicolás et al., 2017). Thus, through the implementation of Virtual Learning Objects, awareness can be generated in managing water resources in the population and sustainable development in the region (Pesare et al., 2015).
The Collaborative Learning methodology based on patterns for the production and use of Learning Objects (MACOBA) has as its pedagogical foundation the student-centered teaching-learning paradigm, which allows it to be applied in many areas of education, in this case, competency-based education (Fuentes et al., 2008; Van Melle et al., 2019), since the methodology is based on the development of software adding the pedagogical component required for the educational process, in addition, it can be implemented in the administration of users and classes in platforms such as Moodle, Exe, among others, added to different programming environments (De Medio et al., 2020), and in obtaining virtual learning objects, based on three fundamental elements: requirements level, design/development level and implementation level (Carrillo Ríos et al., 2019).
2 METHOD
The study conducted is of a cross-sectional type that allowed measuring the impact on the implementation and result of the OVA for the use in the inhabitants of the upper basin of the Bogotá D.C. river evidencing the concepts of apprehension in the management of the water resource, according to the developed instrument (Villasís-Keever et al., 2018).
For the data collection technique in the development of OVA, virtual surveys with closed questions were used as a fundamental basis in the initial phase of the MACOBA methodology
(Margain Fuentes et al., 2009).This was done according to the level of requirements for the development of the software oriented to the contents to be implemented in the software that will allow developing the apprehension of knowledge, adding the pedagogical component to be implemented and the types of evaluation that will be integrated into the software.At the development level, the diagrams and the integration between them were implemented based on the data modeling under UML and the user interface (Tutor, students), and finally, the implementation level, where the prototype was evaluated. Figure 1 allows identifying the different stages of the MACOBA methodology. The order in the development of the methodology stages allows the interaction of different professionals to have a quality product that contributes to the learners and the different professionals involved in the development of a virtual learning object.
The implementation was evaluated under three stages; the first stage was evaluated under the level of coherence that the tool must have, verified by the application developer personnel. In the second stage, information is collected about the use of the application in advanced users who know computational tools and the problem generated in the target population, and in the last stage of evaluation, it is understood what the final user can expect from the management of the water resource. According to the study, a representative sample would be taken to give weight and value to the software from the user and developer's point of view.
The following variables were considered in the research results to carry out the evaluation process:
Study population: Men and women between 20 and 30 years of age.
Stage 1 and 2: 20 subjects, 10th semester systems engineering students at the Universidad Cooperativa de Colombia.
Stages 3 and 4 were divided into 5 evaluation dimensions distributed in the surveys by groups of questions to evaluate the perception of the developer and user with knowledge of computer tools but who also knew the problem situation.
The educational, content, aesthetic, functional and accessibility dimensions were evaluated.
-P. htrAmu (fv«4vcc*n|
__r. Apttnduij« ICanwmo)
Figure 1. Macoba Methodology(Margain Fuentes et al., 2009)
3 RESULTS
1. Las actividades educativas disponibles poseen instrucciones para su desarrollo
2. Las instrucciones proporcionadas son comprensibles
3. La herramienta retroalimenta sobre los errores cometidos en las actividades
4. Las imágenes utilizadas en las actividades y en la aplicación en general son acertadas con la realidad de lo que quiere representar
16
Figure 2, Educational Dimension Results (Garzon Solano et al., 2019)
Figure 2 shows questions 1 to 4 posed to validate the educational dimension. As a result, 95% of the respondents stated that the tool complies with the educational levels, but the remaining 5% indicated that the images should be improved.
5. La arquitectura general y la jerarquía de los contenidos tienen sentido.
6. Se usa el idioma de forma adecuada y estándar, siendo entendible para todo tipo de usuario,
7. El contenido presenta un buen uso de la gramática. S. El contenido presenta un buen uso de la ortografía.
9. Es posible acceder a los contenidos o recursos educativos sin crear un usuario.
• Si
• ™
0 NoApIta
Figure 3.Results Content Dimension (Garzon Solano et al., 2019).
Questions 5 to 9 in Figure 3 were designed to evaluate the content dimension of the ICT tool, and it was observed that 100% of the respondents indicated that the content was simple and in line with the topic; in essence, it allows learning and understanding of the topics of the main objective.
10. Hay suficiente contraste entre el texto y su color de fondo.
11. El contraste de los colores es estéticamente agradable.
n
i rg 3
0 MoApiiu 0
£ CfM&M 0
12. Los enlaces y botones son visualmente i den tífica bles, • a
0 no Q toa Ap
13. La distribución y tamaño de los elementos es adecuada,
14. La tipografía usada para el texto es legible y permite una adecuada velocidad de lectura.
4 Si 18
f re Z
0 № Api ta 0
Figure 4. Results Aesthetic dimension (Garzon Solano et al., 2019).
Questions 10 to 14 of Figure 4 allow the evaluation of the aesthetic dimension, and according to the results obtained, it is evident that the tool has a level of difficulty regarding the comprehension of the interface with 90% approval. However, the evaluators consider that this dimension could be approved since what is sought is learning and not presentation.
Figure 5.Results Functional dimension (Garzon Solano et al., 2019).
Questions 15 to 30 presented in Figure 5 allow evaluating the functional dimension, 5 and according to the results, it is found that 90% comply with the operational objective for the different processes carried out in the basin in terms of water resource management. However, it is evident in the results that the instructions are not clear enough. This result is represented in 5% of the surveyed population, and 95% indicate that they comply.
Figure 6.Accessibility Dimension Results (Garzon Solano et al., 2019).
Finally, in Figure 6, questions 30 to 45 were designed to evaluate the essential elements for accessing the tool and the indications to both users and administrators of the application. The evaluation evaluated the speed of user navigation within the tool, ease and management of the interface, and it was obtained that 85% of respondents indicated that it complied satisfactorily, compared to 5% where it did not meet this feature, on the other hand, the evaluators added 10% as not applicable to the tool since this last item was not taken into account.
The following are the results of the survey applied to advanced users and population experts, which consists of 7 questions to evaluate the tool.
Figure 7.Results Stage 3 advanced users and population experts (Garzon Solano et al.,
2019).
According to the results obtained in Figure 7, the evaluator establishes that the ease of use can positively impact the vast majority of the population since 90% of these users agree that the tool is easy to use.
The following image shows the results obtained by the evaluator.
Easy tool of the tool •90% agree • 10% Disagree
Causes of interaction of the populationwith the tool • Understanding of how to use the tool • Lack of knowledge
Use of the tool for learning about water resource management. •90% Agree • 10% Disagree
90% agree • 10% Disagree
Aspects of tool improvement •Manipulation
Aspects of graphic enhancement
• Interface
Figure 8. Analysis of results in stage 3 (Garzon Solano et al., 2019).
As shown in Figure 8, the analysis results from stage 3, the evaluator performed a percentage analysis for each of the cases and concluded that the design, applicability, ease and understanding in 90% agree and that this can benefit the community in training for the management of water resources.
The results obtained by dimensions were as follows:
1. La herramienta puede contribuir a los procesos de aprendizaje de la población a la que se dirige?
2. Hay intención de volver a utilizar esta aplicación?
Promwor«
Pativos
OaínctDiw
-ICO*
NPS*
3. Se recomienda esta herramienta a otras personas de la población objetivo
Promotor« Detractw»
¡0*
4. La herramienta cumple su objetivo.
Promowres 17
Píliww 3
DatrictorK 0
- "
5. Los elementos multimedia utilizados en la herramienta facilitan el proceso de enseña nza/aprend izaje.
Promotor«
Pasiv«
Detracto1«
■ ■ m ■
Figure 9.Results educational dimension in stage 4 (Garzon Solano et al., 2019).
In Figure 9, questions 1 to 5 are presented to evaluate this dimension, according to the evaluator, indicating a fairly high level of acceptance since 75% of the respondents stated that it contributed to the learning processes, according to the answers to question 1.
The other responses evaluated the intention to use the application again, whether they considered that it would benefit other inhabitants outside the town, and finally, the improvement of the educational content.
6. El co-iter¡do presentado es claro y coherente, Prafrcwwf 14
Éaii-.ILK -
□■tract«™; a
7. El coiterido presentado es de csllcad,
Prenderes
Pmíhw 6
o
0
f«O
NPS«
-1MP "-I«
NP$e
3. IEII contenido presentado presenta una adecuada gramática y ortografía.
Prgmqmgrti
PaaivoB
Dnrien»
-loa* loo
NPSÍ
Figure 10. Results dimension content in stage 4 (Garzon Solano et al., 2019).
Questions 6 to 8 shown in Figure 10 were classified for the evaluation of the content of the tool. In Figure 9, it is evident that for the respondents, the contents are clear and coherent and oriented to the water resource management procedures. The evaluator indicates that 80% agree with question 6, but 70% indicate that in terms of quality, it complies with the requirements of the tool.
9. La elección y descripción de imágenes, sonidos, colores y otros elementos multimedia es adecuada,
Promos««
10, La interfaz de la herramienta es agradable,
Premowre
PM¡vw
Detracto*»
NPS*
A
o*
NPS®
11. La tipografíaj el color y el tamaño utilizado para el texto es legible permitiendo una adecuada velocidad de lectura
Promotor« Paiivra
Detf saches
o"
MPS®
Figure 11. Results aesthetic dimension in stage 4 (Garzon Solano et al., 2019).
Questions 9 to 11, shown in Figure 11, are taken to evaluate the aesthetic dimension of the tool, and the results show that it complies with the combination of environmentally oriented colors.
According to the evaluator, the tool is effectively articulated with the environment in which it is being applied since the user concatenates the colors and the environment in which they visualize daily and compares it with the environment of the tool on which they are focused.
12. La interfaz de la herramienta es fácil de Lsar,
^awos
13, Las instrucciones presentadas fueron suficientes y claras para permitir su adecuada navegación en la herramienta.
Swaaor«
14, Los procesos que se llevaron a cabo con la herramienta son fáciles para el usuario.
^romowm
Parvos
DftracTOf«
15. El rendimiento de la herramienta fue el adecuado para permitir su interacción cor la misma. o
Sramotarw
Detractor«
Figure 12. Results functional dimension in stage4 (Garzon Solano et al., 2019).
Questions 12 to 15 are taken to evaluate the functional dimension for the visual environment that the user has and are shown in Figure 12, taking as a conclusion that according to the selected sample, the interface complies in a significant way according to the knowledge that the population of the basin has, however, the difficulties presented were evaluated the evaluator determined that they were minimal due to the clear handling of the instructions given.
16. La herramienta presenta audios o sintetizado res de voz y estos son claros y eníendibles.
17. La estructura que presenta la herramienta facilitó su navegación.
Prsrowiw
Pasivos
Otínactcm;
16. La estructura y accesibilidad de la herramienta le permitieron! terminar las actividades propuestas en la prueba f actividades educativas. 19. La herramienta le permitió hacer uso de todos ios enlaces, botones, listas desplegabas y similares que usted deseó utilizar
Pnomowes 13
Pwivos 2
DfltrxtorH 0
bp
NPSJ
Figure 13. Results of accessibility dimension in stage 4 (Garzon Solano et al., 2019).
Finally, Figure 13 shows the results of questions 16 to 19, which were designed to evaluate the essential elements for accessing the tool and the indications to both users and administrators of the application, and evaluated the speed of user navigation within it, 85% of the respondents indicated that it complied satisfactorily, while 5% did not. On the other hand, the evaluators added 10% as not applicable to the tool since this last item was not considered.
4 Discussion
It can be identified that according to the target population, the results are very positive in terms of functionality and user perception. The fieldwork gave a clearer idea of the needs of the population inhabiting this Basin regarding the management and consumption of water resources. The application of the tool to the selected population allowed visualizing a positive impact on the management and consumption of water resources by the inhabitants of the upper basin of the Bogotá River. This validates the evaluation processes with the MECOBA methodology and the water resource management proposed by Villasis (Villasis-Keever et al., 2018).
5 Conclusions
According to the surveys, the basic requirements determined in the fieldwork carried out by the upper Bogotá River Basin environmental engineers were measured.These requirements were developed with the population living in the upper Bogotá River Basin in mind, without forgetting the impact that can be generated in the middle and lower basins of the same river ("Experiencias y Contribuciones Del CATIE Al Manejo y Gestión de Cuencas Hidrográficas En América Tropical," 2017), many of the contents that were implemented in the tool are mainly focused on water resource management and on the levels of contamination that the discharge of wastewater can generate into the riverbed that mainly affects flora and fauna and logically the population found in the following sections of the River. This is how the fieldwork gave a clearer idea about the needs of the population living in this basin for water management and consumption, and it is also mainly oriented to the verification of the current processes that are being carried out for this procedure and to achieve the reuse of water, minimizing waste and the best treatment of the water that is discharged into the Bogotá riverbed.
When implementing the web application, according to the results obtained, there is evidence of a fairly high degree of acceptance in the population to which the tool was applied. This is demonstrated in the results obtained in advanced users in systems and in web platform management. Although there were some shortcomings in the presentation interface of the tool, as in the evaluations, it also shows a positive impact on the management and visual environment that the users had and developed for the education that needed to be implemented in the Basin.
According to the results obtained, some important points were identified during the methodological process that could be considered for future studies or more precise evaluations. For this reason, the following methodological suggestions identified during the research process are made:
Regarding the selection of the sample, it is suggested first to expand the sample in each of the stages, especially in the last stage, which is aimed at the end user, to generate inclusion and exclusion criteria according to each of the proposed methodology stages [stage 1, 2, 3 and 4], not to use convenience sampling to give more weight to the research.
For the measurement instrument used, it is recommended to define whether it should be self-administered or guided by an interviewer; it is important to provide further reliability to the instrument by determining the intra- and inter-evaluator error.
Finally, in the presentation of the results, it is suggested that in order to obtain more quantitative data and be able to present descriptive data, it is important to characterize the population by age, sex, academic level, stratum and occupation.
It would be important to implement a pilot test of the survey to reduce research biases and give it greater weight, as well as to present levels of association between the questions posed within each of the dimensions by stages.
And for the presentation of the final results, it would be convenient to make frequency tables according to the population characteristics [age, sex, academic level, stratum, occupation] for each of the dimensions by stages since the results for this case were presented in a general way without classifying the sample.
References
[1] Barari, N., RezaeiZadeh, M., Khorasani, A., & Alami, F. (2020). Designing and validating educational standards for E-teaching in virtual learning environments (VLEs), based on revised Bloom's taxonomy. Interactive Learning Environments, 0(0), 1-13. https://doi.org/10.1080/10494820.2020.1739078
[2] Burgos-Ayala, A., Jiménez-Aceituno, A., & Rozas-Vásquez, D. (2020). Integrating Ecosystem Services in Nature Conservation for Colombia. Environmental Management, 66(2), 149-161. https://doi.org/10.1007/s00267-020-01301-9
[3] Carrillo Ríos, S. L., Tigre Ortega, F. G, Tubón Nuñez, E. E., & Sánchez Villegas, D. S. (2019). Objetos Virtuales de Aprendizaje como estrategia didáctica de enseñanza aprendizaje en la educación superior tecnológica. RECIMUNDO, 3(1), 287-304. https: / /doi.org/10.26820/recimundo/3.(1). enero. 2018.287-304
[4] Cóndor-Herrera, O., Jadán-Guerrero, J., & Ramos-Galarza, C. (2021). Virtual Learning Objects' of Math Educative Process. Advances in Intelligent Systems and Computing, 1269 AISC, 192-197. https: / /doi.org/10.1007/ 978-3-030-58282-1_31
[5] De Medio, C., Limongelli, C., Sciarrone, F., & Temperini, M. (2020). MoodleREC: A recommendation system for creating courses using the moodle e-learning platform. Computers in Human Behavior, 104, 106168. https://doi.org/10.1016/j.chb.2019.106168
[6] Díaz-Casallas, D. M., Castro-Fernández, M. F., Bocos, E., Montenegro-Marin, C. E., & Crespo, R. G. (2019). 2008-2017 Bogota River water quality assessment based on the water quality index. Sustainability (Switzerland), 11(6), 1668. https://doi.org/10.3390/su11061668
[7] Experiencias y contribuciones del CATIE al manejo y gestión de cuencas hidrográficas en América tropical. (2017). Revista de Ciencias Ambientales, 53(1), 153. https://doi.org/10.15359/rca.53-1.9
[8] Fuentes, L. M., Arteaga, J. M., & Rodríguez, F. Á. (2008). A methodology for design collaborative learning objects. Proceedings - The 8th IEEE International Conference on Advanced Learning Technologies, ICALT 2008, 87-91. https://doi.org/10.1109/ICALT.2008.319
[9] Garzon Solano, J. E., Diaz Cabiativa, E. C., jimmy.garzon@campusucc.edu.co, & edgar.diaz@campusucc.edu.co. (2019). Desarrollo de una herramienta TIC basada en sistemas web para la gestión de objetos virtuales de aprendizaje (OVA) que apoye cambios de hábitos en el manejo y consumo del agua, a los habitantes de la Cuenca Alta del río Bogotá. ANDES, U. D. L. (2015). UNIVERSIDAD DE LOS ANDES GERENCIA DEL CAMPUS MANUAL DE ESTÁNDARES. Retrieved from Https: / / Campusinfo. Uniandes. Edu. Co/Images/Stories/Campus/Recursos_fisicos/Manual_estandares_
2015.Pdf. https://repository. ucc. edu. co/handle/20.500.12494/13657
[10] Muratoglu, A. (2019). Water footprint assessment within a catchment: A case study for Upper Tigris River Basin. Ecological Indicators, 106. https://doi.org/10.1016/j.ecolind.2019.105467
[11] Nicolás, C., Pava, R., Gabriel, A., Sanabria, Z., Consuelo, L., & Leal, S. (2017). Articulo original producto de la investigación. NOVA, 15(27), 119-129. http://www.scielo.org.co/pdf/nova/v15n28/1794-2470-nova-15-28-
00069.pdf%0Ahttp://www.scielo.org.co/pdf/nova/v15n27/1794-2470-nova-15-27-00119.pdf
[12] Nohora, M. A., & Moreno García, P. (2017). The implementation of a virtual learning object (VLO) 2 A VIRTUAL LEARNING OBJECT (VLO) AS A TOOL TO TEACH READING FOR SPECIFIC PURPOSES TO COMMUNICATION AND JOURNALISM STUDENTS AT FUNDACIÓN UNIVERSITARIA UNINPAHU SANDRA CECILIA HERNÁNDEZ URREGO.
[13] Pesare, E., Roselli, T., Rossano, V., & Di Bitonto, P. (2015). Digitally enhanced assessment in virtual learning environments. Journal of Visual Languages and Computing, 31, 252-259. https://doi.org/10.1016/j.jvlc.2015.10.021
[14] Ramírez, F., & Santana, J. C. (2019). Key lessons learned by teaching ecotourism to undergraduate students in Bogotá's urban wetlands. Applied Environmental Education and Communication, 18(3), 234-251. https://doi.org/10.1080/1533015X.2018.1454359
[15] Salvador, P. T. C. de O., Bezerril, M. D. S., Mariz, C. M. S., Fernandes, M. I. D., Martins, J. C. A., & Santos, V. E. P. (2017). Virtual learning object and environment: a concept analysis. In Revista brasileira de enfermagem (Vol. 70, Issue 3, pp. 572-579). https://doi.org/10.1590/0034-7167-2016-0123
[16] Sanabria-Suarez, A. C., Montenegro-Marin, C., Castro-Fernandez, M. F., & Diaz-Casallas, D. M. (2017). Multitemporal analysis of the water quality index (wqi) in surface currents in the upper basin of the bogota river (colombia)/analisis multitemporal de los indicadores de calidad de agua en corrientes superficiales (ICA) de la cuenca alta del Rio Bogota (. Revista Ingenieria Solidaria, 13(22), 39-55.
https://go.gale.com/ps/i.do?p=IFME&sw=w&issn=19003102&v=2.1&it=r&id=GALE%7CA585718615&sid =googleScho lar&linkaccess=fu lltext
[17] Sánchez-Alfonso, A. C., Venegas, C., Díez, H., Méndez, J., Blanch, A. R., Jofre, J., & Campos, C. (2020). Microbial indicators and molecular markers used to differentiate the source of faecal pollution in the Bogotá River (Colombia). International Journal of Hygiene and Environmental Health, 225. https://doi.org/10.1016/j.ijheh.2020.113450
[18] Van Melle, E., Frank, J. R., Holmboe, E. S., Dagnone, D., Stockley, D., & Sherbino, J. (2019). A Core Components Framework for Evaluating Implementation of Competency-Based Medical Education Programs. Academic Medicine, 94(7), 1002-1009. https://doi.org/10.1097/ACM.0000000000002743
[19] Villasís-Keever, M. Á., Márquez-González, H., Zurita-Cruz, J. N., Miranda-Novales, G., & Escamilla-Núñez, A. (2018). Research protocol VII. Validity and reliability of the measurements. Revista Alergia Mexico, 65(4), 414-421. https://doi.org/10.29262/ram.v65i4.560
[20] Y. Margain Fuentes, M. de L., Rodríguez Álvarez, F. J., & Arteaga Jaime, M. (2009). Metodología de Aprendizaje Colaborativo fundamentada en patrones para la producción y uso de Objetos de Aprendizaje. Investigación y Ciencia, 22-28. http:// redalyc. uaemex. mx/src/inicio/ArtPdfRed.jsp?iCve=67411476005
