CC BY
66
Original scientific paper UDK:
378.091:004
Received: August, 13.2022. 378.147::57
Revised: November, 11.2022. d 10.23947/2334-8496-2022-10-3-129-138
Accepted: November, 16.2022. ^ CheQk (or updates
Students' Acceptance of Mobile Augmented Reality Applications in Primary and Secondary Biology Education
Ivan Stojsic1' , Natalija Ostojic2 , Jelena Stanisavljevic3
University of Belgrade, Faculty of Biology, Centre for Educational Technology, Didactics' Training and Career Guidance of Biology Teachers, Belgrade, Serbia, e-mail: [email protected] 2Sabac Gymnasium, Sabac, Serbia, e-mail: [email protected] 3University of Belgrade, Faculty of Biology, Belgrade, Serbia, e-mail: [email protected]
Abstract: Augmented reality is often indicated as a usable educational technology that can be integrated into biology classes to overcome the shortcomings of traditional teaching (such as lack of visualization of abstract teaching content, students' low participation and interest in classes, and their insufficient understanding of complex topics). Mobile applications with augmented reality experience mode have the potential to be used in online, blended/hybrid, and in-person teaching, which is particularly important during emergencies. This study's purpose was to determine primary and secondary school students' acceptance of augmented reality content in commercial mobile applications that can be used as a supplement in biology teaching. A total of 188 students (from schools included in this research) completed the online questionnaire. The results showed that the majority of students perceived mobile augmented reality applications as useful and easy to use, had a positive attitude, and expressed intention to use this educational technology if given the opportunity. The importance of prior evaluation regarding educational usability and performance is highlighted since technical quality (of used mobile applications) had a strong positive effect on perceived usefulness and perceived ease of use. There were no statistically significant differences between female and male and primary and secondary students, but students with prior experience with augmented reality rated perceived usefulness higher. Despite positive results, we need to raise our concerns regarding the reliability of using mobile augmented reality in biology education due to the lack of usable free content and the frequent cancellation of authoring tools and applications.
Keywords: augmented reality, biology teaching, mobile application, Technology Acceptance Model, technology-enhanced learning.
Introduction
A lot of biology learning materials are abstract and difficult to understand due to the complexity of life concepts, especially if the learning content is microscopic or not available for direct observation (Chang, Chung and Huang, 2016; Nurhasanah, Widodo and Riandi, 2019; Wang et al., 2022). Therefore, digital visualization technologies have become essential for biology education since special equipment (such as high-tech microscopes) is often not affordable for educational institutions (Erbas and Demirer, 2019; Jenkinson, 2018).
The current COVID-19 pandemic highlighted the need for digital resources that can be used in blended, hybrid, and online K-12 teaching (Crompton et al., 2021). Still, in order to successfully organize technology-enhanced learning solutions (e.g., using digital technologies for presenting knowledge differently, creating active hands-on learning activities, and providing solutions for evaluation of acquired knowledge), teachers need to have skills in using digital tools, as well as a certain level of smart pedagogical competences (Daniela, 2021).
Despite much broader availability of immersive technologies (such as augmented reality [AR] and virtual reality [VR]) to educational institutions in recent years, adoption is lagging. However, due to the ongoing pandemic, the potential of using VR and AR content in blended/hybrid learning has been emphasized (Garcia Estrada and Prasolova-Forland, 2022).
In the literature (Chang, Chung and Huang, 2016; Chien et al., 2019; Erbas and Demirer, 2019;
'Corresponding author: [email protected] |© 2022 by the authors. This article is
¡Creative Commons Attribution (CC BY) license (https://creativecommons.Org/licenses/by/4.0/).
^ | © 2022 by the authors. This article is an open access article distributed under the terms and conditions of the
Fuchsova and Korenova, 2019; Hung, Chen and Huang, 2017; Hwang et al., 2016; Jenkinson, 2018; Lu and Liu, 2015; Safadel and White, 2019; Wang et al., 2022; Weng et al., 2020; Yapici and Karakoyun, 2021), AR is often indicated as a relatively new and usable technology for biology (including ecology) teaching and learning at all levels of education. Yavuz et al. (2021) pointed out mobile AR (MAR) applications (apps) as affordable and sustainable for massive adoption in different areas (including education). According to Laine (2018, p. 2), MAR can be defined as "a type of AR where a mobile device (smartphone or tablet) is used to display and interact with virtual content, such as three-dimensional (3D) models, annotations, and videos, that are overlaid on top of a real-time camera feed of the real world". Using mobile devices to integrate AR content in in-person or remote educational settings is more accessible and less expensive than with other types of AR hardware (such as smart glasses, headsets, AR projection systems, etc.) since most students already have appropriate smartphones (Nurhasanah, Widodo and Riandi, 2019).
Although still limited and content-specific (Erbas and Demirer, 2019), the body of literature concerning the use of AR in biology (including ecology) education is constantly growing. On the one hand, several studies have shown that AR can positively affect students' achievement in biology and ecology in formal and informal learning settings (Hwang et al., 2016; Lu and Liu, 2015; Nurhasanah, Widodo and Riandi, 2019). On the other hand, a number of studies didn't find any significant difference in students' academic achievement (learning outcomes) between AR and traditional learning materials or other digital aids (Chang, Chung and Huang, 2016; Chien et al., 2019; Erbas and Demirer, 2019; Hung, Chen and Huang, 2017; Wang et al., 2022; Weng et al., 2020). Still, Chang, Chung and Huang (2016) reported better knowledge retention, Wang et al. (2022) reported a reduction in students' cognitive load, and Chien et al. (2019) and Weng et al. (2020) reported statistically higher scores in the experimental group (that used the aR technology) on questions related to higher levels of Bloom's revised taxonomy (such as the level of analyzing). Also, the majority of analyzed studies reported benefits of AR regarding students' motivation, self-efficacy, satisfaction, and/or participation in biology lessons.
According to Nurhasanah, Widodo and Riandi (2019, p. 482), the use of AR in biology classes "will undoubtedly attract more students' interest in school". Similarly, Hung, Chen and Huang (2017) pointed out that AR may not be superior compared to other aids and teaching materials, but it is at least equally effective and can help students learn biology, spark their interest, and reduce classroom boredom. In addition, Lu and Liu (2015) emphasized that learning activities with AR can be especially helpful for low academic achievement students, and Chang, Chung and Huang (2016) indicated students' opportunity to experience constructivist learning as one of the most important advantages of using AR in schools.
Dengel et al. (2022) pointed out that teachers should be capable of designing their AR experiences and indicated five accessible AR authoring toolkits for educational purposes (Vuforia Studio, BlippAR, AWE, AR Media Studio, and Areeka). However, teachers often lack specific knowledge and skills to develop or customize their own digital materials, such as AR experiences (Daniela, 2021; Mota et al., 2018). Also, Daniela (2021, p. 714) emphasized that it is not clear "how much effort the teacher should put into developing the materials". Fuchsova and Korenova (2019) suggested a few commercial biology-themed MAR apps that are affordable and appropriate for teaching. Still, using so-called "off-the-shelf" apps in learning environments is not a straightforward process since available immersive experiences need to be evaluated first and matched with teaching content and lesson goals (Stojsic et al., 2019b). For example, Dreimane and Daniela (2021) analyzed 41 MAR apps (from the App Store) related to the anatomy of the human body, but only seven met the selection criteria. The same authors believe that commercial MAR apps can be successfully integrated into the learning process (but before all else teachers need to understand the educational potential and limitations of those apps) and proposed an evaluation framework with 19 criteria divided into three groups: (a) technological performance, (b) information architecture, and (c) educational value (Dreimane and Daniela, 2021).
Besides the availability of MAR apps (both custom and off-the-shelf) to biology teachers and students and their usability, graphics quality, and effectiveness, it is also important to assess the acceptance of those apps. According to Yavuz et al. (2021, p. 1), acceptance of MAR is "one of the factors influencing its adoption". Yapici and Karakoyun (2021) conducted a case study with prospective biology teachers and the result showed that future biology teachers had mostly positive views about the use of AR in biology teaching. Although limited in scope, previous studies (Fuchsova and Korenova, 2019; Hung, Chen and Huang, 2017; Hwang et al., 2016; Safadel and White, 2019) also suggested that the majority of students accept AR and have positive attitudes (perceptions) regarding the use of this technology for biology learning. However, we did not find studies that deal with determining primary and/or secondary students' acceptance of commercially available MAR apps (for biology learning) during pandemic teaching.
The purpose of the study and used research model
The purpose of this study was to determine primary and secondary students' acceptance of free commercial MAR apps that can be used as a supplement in biology teaching/learning, as well as to identify potential variables that influence acceptance.
In the literature (Balog and Pribeanu, 2010; Cabero-Almenara, Fernández-Batanero and Barroso-Osuna, 2019; Huang and Liaw, 2018; Huang, Liaw and Lai, 2016; Mailizar and Johar, 2021; Wojciechowski and Cellary, 2013), students' acceptance of AR, VR, and other immersive technologies in educational settings was often researched using the Technology Acceptance Model (TAM; Davis, 1989; Davis, Bagozzi and Warshaw, 1989). According to Trivunovic and Kosanovic (2021), the TAM model provides insights into the reasons for acceptance and use of technology in teaching and learning processes. In other words, the TAM constructs (factors) explain the complexity of the process of technology acceptance by the user (in our case the student).
In the present study, we used an adjusted and shortened version of the AR Acceptance Model (based on the TAM model) proposed by Cabero Almenara, Barroso Osuna and Llorente Cejudo (2016). On the basis of the used research model (Figure 1), the following hypotheses were formulated:
H1. Technical quality has a positive effect on perceived usefulness.
H2. Technical quality has a positive effect on perceived ease of use.
H3. Perceived ease of use has a positive effect on perceived usefulness.
H4. Perceived ease of use has a positive effect on attitude toward use.
H5. Perceived usefulness has a positive effect on attitude toward use.
H6. Perceived usefulness has a positive effect on intention to use.
H7. Attitude toward use has a positive effect on intention to use.
Figure 1. Research model (based on Cabero Almenara, Barroso Osuna and Llorente Cejudo,
2016).
In addition, but in line with the research purpose, we formulated the following research questions:
RQ1. Does gender influence differences in students' acceptance of MAR apps?
RQ2. Does prior achievement in biology (grade at the end of the first semester) influence differences in students' acceptance of MAR apps?
RQ3. Does the type of school (primary or secondary) influence differences in students' acceptance of MAR apps?
RQ4. Does prior experience with AR influence differences in students' acceptance of MAR apps?
We decided to formulate research questions instead of hypotheses since there are no sufficient and conclusive results from previous studies regarding the influence of investigated variables (gender, biology grade at the end of the first semester, type of school, and prior experience with AR) on students' acceptance of the educational use of AR technology.
Materials and Methods
In this research, mobile apps with AR experience mode were used to complement students' knowledge acquisition of certain biological teaching content. At the time we started our research, there was a need for digital solutions that could help students' better understanding of biology learning materials since the classes were shorted to 30 minutes due to pandemic measures (with only half of the students in the classroom). The research was conducted during the second semester of the 2020-2021 school year through four stages.
In the first stage, we were looking for AR content in free commercial mobile apps that could be used in biology classes. The search didn't include MAR apps that are only for iOS devices (e.g., iPad and iPhone) since in the Republic of Serbia the majority of schools', teachers', and students' owned devices (tablets and smartphones) are Android-based.
In the second stage, we checked the performance and graphics quality of the AR experience mode
(of biology-related content) in found MAR apps, as well as used the heuristic questionnaire proposed by Radu (2014) to evaluate those apps. After the evaluation process, we were left with only a few MAR apps (EON-XR, Expeditions [the app is no longer available], Edmentum AR Biology, and WWF Free Rivers) that we tried to match with the teaching content in different grades of the primary and secondary school biology curriculum.
In the third stage, the integration process was planned and realized following steps from the AR/ VR integration model proposed by Stojsic et al. (2019a). For instance, the steps included evaluation of school infrastructure and availability of necessary devices (teachers' and students' owned smartphones and tablets), as well as taking security measures and preparing students to use MAR apps.
In the fourth stage, the selected biology-related AR content was included in some biology classes (in both in-person and online groups) as an additional part of activities created by the second author (biology teacher). For example, the EON-XR app was employed with the teaching content related to nutrition and the human digestive system. At the end of this research (end of the second semester), the students were offered to fill in an online questionnaire. Participation was voluntary and anonymous.
Participants
Three state primary schools (two rural and one urban) and one urban state secondary school in the Republic of Serbia took part in this research. A total of 188 students (from schools included in this research) completed the online questionnaire correctly and timely. Participants' characteristics are presented in Table 1.
Table 1
Participants' descriptive characteristics (N = 188)
Variables n %
Gender
female 111 59.04
male 77 40.96 Biology grade
(at the end of the first semester)
insufficient (1) 1 0.53
sufficient (2) 15 7.98
good (3) 24 12.77
very good (4) 53 28.19
excellent (5) 95 50.53 Type of school
primary 127 67.55
secondary 61 32.45 Prior experience with AR
yes 103 54.79
no 85 45.21
Instrument
An online questionnaire in the Serbian language was created (with Google Forms) as the instrument for this research. The first part of the questionnaire contained questions related to students' demographics and school information (e.g., gender, type of school, biology grade at the end of the first semester, etc.). The second part of the questionnaire included a five-point Likert-type scale (from 1 - strongly disagree to 5 - strongly agree) with 19 items grouped to measure five TAM constructs (technical quality, perceived usefulness, perceived ease of use, attitude toward use, and intention to use). The items were defined as positive and negative statements and mostly adapted from Cabero Almenara, Barroso Osuna and
Llorente Cejudo (2016). Table 2 shows Cronbach's alpha values for TAM constructs and means and standard deviations of items.
Table 2
Cronbach's alpha values for TAM constructs and means and standard deviations of items
TAM constructs Items M SD Cronbach's alpha
3D objects in MAR apps provide a sense of reality. 4.09 0.36
Technical quality Objecls and scenes seen in AR mode are aesthetically pleasing. 4.05 0.76 .83
AR content in used mobile apps is attractive. 3.S5 0.90
MAR apps helped me master (he learning material. 3.95 0.94
The 3D view in MAR apps helped me better understand the
structure of a biological system [e.g, the human digestive 4.25 0.87
system).
Perceived usefulness Thanks to the use of MAR apps, I learned more compared to traditional ciasses. 3.71 1.10 .83
My attention in biology classes is better when (here is additional AR content. 3.65 1.10
Using MAR apps could Improve biology learning in the classroom {or online). 4.04 0.91
Perceived ease 1 thintt that MAR apps are easy fo use. 4.24 0.71 .67
of use It was not a problem lor me to learn how to use a MAR app. 4.43 0.66
Using MAR apps In biology cfasses makes learning fun. 4.20 0.E7
Attitude toward use AR content makes biology learning mone interesting. 'Learning biology wltfi MAR apps is ijoring. 4.23 3.91 0.77 1.12 .80
1 believe lhat using MAR apps In Itie classroom (or in online activities) is a good idea. 4.10 0.99
If 1 have the opportunity in lite future, 1 would ilke to use MAR apps for biology learning. 4.05 0.95
Intention to use 1 would like to use MAR apps in oiher school subjects as we 1. *lt is not necessa^ for me to use MAR apps in future Slology classes. 4.15 3.34 0.92 1.17 75
Tm not interested in using MAR apps for learning. 3.75 1.13
Note. The item "I had a hard time mastering the use of MAR apps." was excluded from the perceived ease of use construct due to low item-total correlation. For the negative items (marked with an asterisk [*]), a reverse scoring method was used.
Data Analysis
Statistical analyses were performed using IBM SPSS Statistics software (version 24). Besides descriptive statistics (arithmetic means, standard deviations, frequencies, and proportions), path analysis (based on multiple regression analysis) was used for testing the research model (hypotheses 1-7). Additionally, four MANOVA (multivariate analysis of variance) tests were run to examine the relationship between independent variables (gender, biology grade at the end of the first semester, type of school, and prior experience with AR) and dependent variables (the TAM constructs).
Results
Analyzing the results from the TAM-based scale (Table 3), it can be concluded that students included in this research accepted the use of MAR apps as a supplement in biology teaching/learning. According to students, MAR apps had sufficient technical quality (M = 4.00, SD = 0.73), they were easy to use (M = 4.34, SD = 0.60), and mostly useful for biology learning (M = 3.93, SD = 0.76). Also, the students
had a positive attitude toward the use of MAR apps in biology teaching (M = 4.12, SD = 0.75) and to a certain degree, expressed their intention to use (M = 3.83, SD = 0.80) this educational technology in the future (if given the opportunity).
Table 3
Means and standard deviations of TAM constructs
TAM constructs M SD
Technical quality 4.00 0.73
Perceived usefulness 393 0.76
Perceived ease of use 4.34 0.60
Attitude toward use 4.12 0.75
Intention to use 3.83 080
To test hypotheses, the path analysis based on multiple regression analysis was used. Therefore, path coefficients are standardized beta values. The results of the path analysis (Table 4) have revealed statistical significance of all paths in the tested research model.
Table 4
Results of the research model testing
Hypotheses Paths Standardized coefficient (ß) t P
H1 Technical quality —► Perceived usefulness 062 10.73 <.001
H2 Technical quality —> Perceived ease of use 046 7.03 <.001
H3 Perceived ease of use —> Perceived usefulness 0.16 2.76 .006
H4 Perceived ease of use —» Attitude toward use 012 2.04 .043
H5 Perceived usefulness ^Attitude toward use 0.62 10.41 <.001
H6 Perceived usefulness -»■ Intention to use 018 2.76 .005
H7 Attitude toward use Intention to use 062 0.54 <.001
The path between perceived ease of use and attitude toward use was found significant at a .05 level, whereas the paths between perceived ease of use and perceived usefulness and perceived usefulness and intention to use were found significant at a .01 level. The rest of the paths were found significant at a .001 level. Consequently, all hypotheses are supported. Figure 2 shows the final model.
Figure 2. Results of the path analysis (Final model).
To answer formulated research questions, four MANOVA tests were performed. Preliminary tests were conducted to check violations for normality, linearity, univariate and multivariate outliers, homogeneity, and multicollinearity assumptions (see Pallant, 2020).
The first research question investigates potential differences in students' acceptance of MAR apps regarding gender (independent variable). The TAM constructs (technical quality, perceived usefulness, perceived ease of use, attitude toward use, and intention to use) were used as dependent variables. During preliminary testing, three cases were removed due to multivariate outliers. The results of the MANOVA test showed no significant difference between female and male students on the combined dependent variables, F(5, 179) = 2.05, p = .073, Pillai's Trace = .05, partial n2 = .05.
The second research question explores potential differences in students' acceptance of MAR apps regarding biology grades at the end of the first semester (independent variable). The TAM constructs were used as dependent variables. During preliminary testing, three cases were removed due to multivariate outliers. Also, the "insufficient" group was excluded because there was only one case in it. Due to the limited number of cases, groups "sufficient" and "good" were merged into one group. The results of the MANOVA test showed a statistically significant difference between students with different biology grades ("sufficient/good", "very good", and "excellent") on the combined dependent variables, F(10, 356) = 2.14, p = .021, Pillai's Trace = .11, partial n2 = .06. However, when the results for the dependent variables were considered separately (using the sequential Holm-Bonferroni method for alpha level correction), none of the differences reached statistical significance.
The third research question examines potential differences in students' acceptance of MAR apps regarding school type (independent variable). Five dependent variables were used (the TAM constructs). During preliminary testing, three cases were removed due to multivariate outliers. The results of the MANOVA test showed no significant difference between primary and secondary school students on the combined dependent variables, F(5, 179) = 1.99, p = .083, Pillai's Trace = .05, partial n2 = .05.
The fourth research question explores potential differences in students' acceptance of MAR apps regarding prior experience with AR (independent variable). Again, the TAM constructs were used as dependent variables. During preliminary testing, three cases were removed due to multivariate outliers. The results of the MANOVA test showed a statistically significant difference between students with and without prior experience with AR on the combined dependent variables, F(5, 179) = 2.51, p = .032, Pillai's Trace = .07, partial n2 = .07. When the results for the dependent variables were considered separately (using the sequential Holm-Bonferroni method for alpha level correction), the only difference to reach statistical significance was perceived usefulness, F(1, 183) = 8.53, p = .004, partial n2 = .04. An inspection of the mean scores indicated that students with prior experience with AR perceived higher usefulness of MAR apps (M = 4.09, SD = 0.67) than students without prior experience with AR (M = 3.78, SD = 0.76).
Discussion
Using smartphone-supported apps is a way to provide everyone a chance to use AR experiences in the learning process (Dreimane and Daniela, 2021). Therefore, this research deals with primary and secondary school students' acceptance of MAR apps (that can be used as a supplement in biology teaching/learning).
Based on the results, it can be concluded that the majority of students accepted MAR apps which is in line with previous studies related to biology (including ecology) content teaching (Fuchsova and Korenova, 2019; Hung, Chen and Huang, 2017; Hwang et al., 2016; Safadel and White, 2019).
The seven hypotheses were tested using the path analysis. The results showed that technical quality was a very strong predictor of students' perceived usefulness and perceived ease of use. Perceived ease of use had a significant positive effect on students' perceived usefulness of MAR apps. Perceived usefulness was the most important predictor (P = 0.62) of students' attitude toward use regarding MAR apps. Also, perceived ease of use had a significant positive impact (P = 0.12) on students' attitude toward use, which was theorized in the research model but not always the case in prior studies regarding MAR apps (e.g., Koutromanos and Mikropoulos, 2021; Yavuz et al., 2021). Furthermore, attitude toward use was the most influential predictor (P = 0.62) of students' intention to use MAR apps. Similar results were reported in studies (regarding immersive technologies) by Cabero-Almenara, Fernández-Batanero and Barroso-Osuna (2019), Koutromanos and Mikropoulos (2021), and Wojciechowski and Cellary (2013). In addition, perceived usefulness had a significant positive impact (P = 0.18) on students' intention to use MAR apps. Although Huang and Liaw (2018) pointed out that in many studies perceived usefulness has been seen as the most significant predictor of students' intention to use various digital technologies
(e-learning systems, virtual worlds, and VR), we should be mindful that the authors of those studies used different TAM constructs (often without attitude as a construct). Additionally, Wojciechowski and Cellary (2013) did not find a significant effect of perceived usefulness on intention to use in their research.
In addressing research questions 1 and 3, the results of MANOVA tests showed no statistically significant differences regarding gender and school type on students' acceptance of MAR apps. Cabero-Almenara, Fernández-Batanero and Barroso-Osuna (2019) also reported that no significant differences were found regarding the influence of students' gender on the degree of acceptance of AR. However, Pombo and Marques (2020) reported that primary school students perceived higher educational value of the EduPARK mAr game than secondary school students.
In relation to research questions 2 and 4, the MANOVA test results showed statistically significant differences regarding biology grades (at the end of the first semester) and prior experience with AR on the combined dependent variables (the tAm constructs). However, for biology grades, none of the differences reached statistical significance when the dependent variables were considered separately. The results are encouraging since students found AR content useful for learning regardless of their prior achievement in biology. According to Salmi, Thuneberg and Vainikainen (2017), AR is one of the few pedagogical solutions that especially benefits those students who are below average in school achievement. For prior experience with AR, the only difference to reach statistical significance was perceived usefulness indicating that students with prior experience with AR perceived higher usefulness of MAR apps. These results are similar to the findings of Stojsic et al. (2020) and suggest that the positive students' responses were not just products of a novelty effect (see Akgayir and Akgayir, 2017).
Conclusions
The results of this research showed that both primary and secondary students accepted MAR apps and perceived their usefulness, as well as they had a positive attitude toward the use of AR in biology teaching and expressed their intention to use this educational technology more frequently if given the opportunity. Still, we need to take into consideration that the BYOD (bring your own device) model remains the only way for many schools and teachers in the Republic of Serbia to introduce certain digital innovations in teaching practice (Atanaskovic et al., 2022). Furthermore, we should bear in mind that "each AR application is unique, influencing students in specific ways according to its design" (Radu, 2014, p. 1534). Therefore, to ensure a meaningful, effective, and successful integration, app evaluation is a necessary step, as well as using appropriate integration models (such as one proposed by Stojsic et al., 2019a). Also, the importance of prior evaluation of apps was highlighted in the results of this research since technical quality (of used MAR apps) had a very strong positive impact on perceived usefulness and perceived ease of use.
Like in the study done by Dreimane and Daniela (2021), we also finished the evaluation process with a limited number of usable MAR apps. Teachers' access to suitable AR/VR content is a bottleneck when it comes to the broader adoption of immersive technologies in education (Garcia Estrada and Prasolova-Forland, 2022). We should emphasize that the Expeditions app was discontinued in June 2021. Additionally, one of the biology-themed MAR apps used in the study by Fuchsova and Korenova (2019) is no longer available as well. Therefore, usable AR content (in currently available free mobile apps that can be used for biology teaching) is limited and learners cannot always use it independently (due to low information architecture and/or educational value, see Dreimane and Daniela, 2021), but it can be integrated (as a supplement) into activities and teacher-created materials to engage students with the teaching content in classrooms or online. However, the question regarding the reliability of using free MAR apps (as the main option for integrating AR in educational settings) is still open. Utilizing AR authoring tools is not more reliable either. For example, Dengel et al. (2022) also raised questions about reliability since over half of AR authoring tools reported in the scientific articles (43 papers were included in the meta-analysis) were not accessible or discontinued. The same authors pointed out that "having to change to a different Authoring Toolkit after a year or two is tedious and could keep educators putting in the effort of learning how to use such toolkits" (Dengel et al., 2022, p. 9). In addition, in their SWOT analysis, Stojsic et al. (2019a) indicated the rapid obsolescence of mobile devices and the cancellation of authoring tools and apps as threats that could jeopardize the wider adoption of immersive technologies in learning environments.
This research has some limitations. The first limitation relates to the selection process since we only included free Android mobile apps (with AR content about biology). Moreover, selection and evaluation processes can involve aspects of subjectivity, which cannot be fully eliminated using evaluation
questionnaires and frameworks (Dreimane and Daniela, 2021). The second limitation is that we didn't have the means to monitor the actual use of MAR apps in online groups of students. The third limitation is the voluntary response bias (possible differences between students who filled in the questionnaire and those who did not).
Acknowledgements
The authors would like to thank dr. Nina Adanin (Northwest Missouri State University) for her help during the data analysis.
Conflict of interests
The authors declare no conflict of interest.
References
Akgayir, M., & Akgayir, G. (2017). Advantages and challenges associated with augmented reality for education: A systematic
review of the literature. Educational Research Review, 20, 1-11. https://doi.org/10.1016Zj.edurev.2016.11.002 Atanaskovic, M., Stojsic, I., Stanisavljevic, L., & Stanisavljevic, J. (2022). Stavovi ucenika o primeni digitalnog kviza u nastavi biologije u srednjoj skoli [Students' attitudes towards the application of digital quiz in high school biology teaching]. Pedagoska stvarnost, 68(1), 48-63. https://doi.org/10.19090/ps.2022.148-63 Balog, A., & Pribeanu, C. (2010). The role of perceived enjoyment in the students' acceptance of an augmented reality teaching platform: A structural equation modelling approach. Studies in Informatics and Control, 19(3), 319-330. https://doi. org/10.24846/v19i3y201011
Cabero Almenara, J., Barroso Osuna, J., & Llorente Cejudo, M. d. C. (2016). Technology Acceptance Model & realidad aumentada: Estudio en desarrollo [Technology Acceptance Model & augmented reality: Study in progress]. Revista Lasallista de Investigación, 13(2), 18-26. https://doi.org/10.22507/rli.v13n2a2 Cabero-Almenara, J., Fernández-Batanero, J. M., & Barroso-Osuna, J. (2019). Adoption of augmented reality technology by
university students. Heliyon, 5(5), Article e01597. https://doi.org/10.1016peliyon.2019.e01597 Chang, R.-C., Chung, L.-Y., & Huang, Y.-M. (2016). Developing an interactive augmented reality system as a complement to plant education and comparing its effectiveness with video learning. Interactive Learning Environments, 24(6), 12451264. https://doi.org/10.1080/10494820.2014.982131 Chien, Y.-C., Su, Y.-N., Wu, T.-T., & Huang, Y.-M. (2019). Enhancing students' botanical learning by using augmented reality.
Universal Access in the Information Society, 18(2), 231-241. https://doi.org/10.1007/s10209-017-0590-4 Crompton, H., Burke, D., Jordan, K., & Wilson, S. W. G. (2021). Learning with technology during emergencies: A systematic review of K-12 education. British Journal of Educational Technology, 52(4), 1554-1575. https://doi.org/10.1111/ bjet.13114
Daniela, L. (2021). Smart pedagogy as a driving wheel for technology-enhanced learning. Technology, Knowledge and
Learning, 26(4), 711-718. https://doi.org/10.1007/s10758-021-09536-z Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS
Quarterly, 13(3), 319-340. https://doi.org/10.2307/249008 Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical
models. Management Science, 35(8), 982-1003. https://doi.org/10.1287/mnsc.35.8.982 Dengel, A., Zahid Iqbal, M., Grafe, S., & Mangina, E. (2022). A review on augmented reality authoring toolkits for education.
Frontiers in Virtual Reality, 3, Article 798032. https://doi.org/10.3389/frvir.2022.798032 Dreimane, S., & Daniela, L. (2021). Educational potential of augmented reality mobile applications for learning the anatomy of the human body. Technology, Knowledge and Learning, 26(4), 763-788. https://doi.org/10.1007/s10758-020-09461-7 Erbas, C., & Demirer, V. (2019). The effects of augmented reality on students' academic achievement and motivation in a biology course. Journal of Computer Assisted Learning, 35(3), 450-458. https://doi.org/10.1111/jcal.12350 Fuchsova, M., & Korenova, L. (2019). Visualisation in basic science and engineering education of future primary school teachers in human biology education using augmented reality. European Journal of Contemporary Education, 8(1), 92-102. https://doi.org/10.13187/ejced.2019.192 Garcia Estrada, J., & Prasolova-Forland, E. (2022). Improving adoption of immersive technologies at a Norwegian university. In A. Dengel, M.-L. Bourguet, D. Pedrosa, J. Hutson, K. Erenli, D. Economou, A. Peña-Rios, & J. Richter (Eds.), Proceedings of 2022 8th International conference of the Immersive Learning Research Network - iLRN (pp. 347-351). Immer-sive Learning Research Network. https://doi.org/10.23919/iLRN55037.2022.9815954 Huang, H. M., & Liaw, S. S. (2018). An analysis of learners' intentions toward virtual reality learning based on constructivist and technology acceptance approaches. International Review of Research in Open and Distributed Learning, 19(1), 91-115. https://doi.org/10.19173/irrodl.v19i1.2503 Huang, H.-M., Liaw, S.-S., & Lai, C.-M. (2016). Exploring learner acceptance of the use of virtual reality in medical education: A case study of desktop and projection-based display systems. Interactive Learning Environments, 24(1), 3-19. https:// doi.org/10.1080/10494820.2013.817436 Hung, Y.-H., Chen, C.-H., & Huang, S.-W. (2017). Applying augmented reality to enhance learning: A study of different teaching
materials. Journal of Computer Assisted Learning, 33(3), 252-266. https://doi.org/10.1111/jcal.12173 Hwang, G.-J., Wu, P.-H., Chen, C.-C., & Tu, N.-T. (2016). Effects of an augmented reality-based educational game on students' learning achievements and attitudes in real-world observations. Interactive Learning Environments, 24(8), 1895-
1906. https://doi.org/10.1080/10494820.2015.1057747 Jenkinson, J. (2018). Molecular biology meets the learning sciences: Visualizations in education and outreach. Journal of
Molecular Biology, 430(21), 4013-4027. https://doi.org/10.1016/jJmb.2018.08.020 Koutromanos, G., & Mikropoulos, T. A. (2021). Mobile augmented reality applications in teaching: A proposed Technology Acceptance Model. In D. Economou, A. Pena-Rios, A. Dengel, H. Dodds, M. Mentzelopoulos, A. Klippel, K. Erenli, M. J. W. Lee, & J. Richter (Eds.), Proceedings of2021 7th International conference of the Immersive Learning Research Network - iLRN (pp. 273-280). Immersive Learning Research Network. https://doi.org/10.23919/iLRN52045.2021.9459343 Laine, T. H. (2018). Mobile educational augmented reality games: A systematic literature review and two case studies. Computers, 7(1), Article 19. https://doi.org/10.3390/computers7010019 Lu, S.-J., & Liu, Y.-C. (2015). Integrating augmented reality technology to enhance children's learning in marine education. Environmental Education Research, 21(4), 525-541. https://doi.org/10.1080/13504622.2014.911247 Mailizar., & Johar, R. (2021). Examining students' intention to use augmented reality in a project-based geometry learning environment. International Journal of Instruction, 14(2), 773-790. https://doi.org/10.29333/iji.2021.14243a Mota, J. M., Ruiz-Rube, I., Dodero, J. M., & Arnedillo-Sanchez, I. (2018). Augmented reality mobile app development for all.
Computers and Electrical Engineering, 65, 250-260. https://doi.org/10.1016/j.compeleceng.2017.08.025 Nurhasanah, Z., Widodo, A., & Riandi, R. (2019). Augmented reality to facilitate students' biology mastering concepts and
digital literacy. JPBI (Jurnal Pendidikan Biologi Indonesia), 5(3), 481-488. https://doi.org/10.22219/jpbi.v5i3.9694 Pallant, J. (2020). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (7th ed). Routledge. https://
doi.org/10.4324/9781003117452 Pombo, L., & Marques, M. M. (2020). The potential educational value of mobile augmented reality games: The case of Edu-
PARK app. Education Sciences, 10(10), Article 287. https://doi.org/10.3390/educsci10100287 Radu, I. (2014). Augmented reality in education: A meta-review and cross-media analysis. Personal and Ubiquitous Computing,
18(6), 1533-1543. https://doi.org/10.1007/s00779-013-0747-y Safadel, P., & White, D. (2019). Facilitating molecular biology teaching by using augmented reality (AR) and Protein Data Bank
(PDB). TechTrends, 63(2), 188-193. https://doi.org/10.1007/s11528-018-0343-0 Salmi, H., Thuneberg, H., & Vainikainen, M.-P. (2017). Making the invisible observable by augmented reality in informal science education context. International Journal of Science Education, Part B, 7(3), 253-268. https://doi.org/10.1080/2154845 5.2016.1254358
Stojsic, I., Ivkov-Dzigurski, A., Bukicin Vuckovic, S., & Maricic, O. (2019a). Primena prosirene i virtuelne realnosti u nastavi geografije: SWOT analiza i predlog integracije [The use of augmented and virtual reality for geography teaching: Swot analysis and integration proposal]. In E. Borsos, R. Horak, C. Kovacs, & Z. Namesztovszki (Eds.), Book of selected papers of the Hungarian Language Teacher Training Faculty's scientific conferences (Mobility) - 8th International methodological conference (pp. 509-523). University of Novi Sad, Hungarian Language Teacher Training Faculty. Retreived from https://magister.uns.ac.rsfiles/kiadvanyok/konf2019/ConfSubotica2019.pdf Stojsic, I., Ivkov-Dzigurski, A., & Maricic, O. (2019b). Virtual reality as a learning tool: How and where to start with immersive teaching. In L. Daniela (Ed.), Didactics of smart pedagogy: Smart pedagogy for technology enhanced learning (pp. 353-369). Springer. https://doi.org/10.1007/978-3-030-01551-0_18 Stojsic, I., Ivkov-Dzigurski, A., Maricic, O., Stanisavljevic, J., Milankovic Jovanov, J., & Visnic, T. (2020). Students' attitudes toward the application of mobile augmented reality in higher education. Drustvena istrazivanja, 29(4), 535-554. https:// doi.org/10.5559/di.29.4.02
Trivunovic, B., & Kosanovic, M. (2021). Faktori prihvatanja upotrebe tehnologije u visokoskolskoj nastavi: TAM model [Acceptance factors of technology use in higher education teaching process: TAM model]. In V. Katic (Ed.), Zbornik radova TREND 2021: XXVII Skup trendovi razvoja "On-line nastava na univerzitetima" (pp.119-121). University of Novi Sad, Faculty of Technical Sciences. Retreived from http://www.trend.uns.ac.rs/stskup/trend_2021/radovi/T1.2/T1.2-5.pdf Wang, X.-M., Hu, Q.-N., Hwang, G.-J., & Yu, X.-H. (2022). Learning with digital technology-facilitated empathy: An augmented reality approach to enhancing students' flow experience, motivation, and achievement in a biology program. Interactive Learning Environments, 1-17. https://doi.org/10.1080/10494820.2022.2057549 Weng, C., Otanga, S., Christianto, S. M., & Chu, R. J.-C. (2020). Enhancing students' biology learning by using augmented reality as a learning supplement. Journal of Educational Computing Research, 58(4), 747-770. https://doi. org/10.1177/0735633119884213 Wojciechowski, R., & Cellary, W. (2013). Evaluation of learners' attitude toward learning in ARIES augmented reality
environments. Computers & Education, 68, 570-585. https://doi.org/10.1016/j.compedu.2013.02.014 Yapici, i. U., & Karakoyun, F. (2021). Using augmented reality in biology teaching. Malaysian Online Journal of Educational
Technology, 9(3), 40-51. http://dx.doi.org/10.52380/mojet.2021.9.3.286 Yavuz, M., Qorbacioglu, E., Ba§oglu, A. N., Daim, T. U., & Shaygan, A. (2021). Augmented reality technology adoption: Case of a mobile application in Turkey. Technology in Society, 66, Article 101598. https://doi.org/10.1016Zj.techsoc.2021.101598
