CC BY
25
Original scientific paper
Received: September 03, 2023. Revised: November 07, 2023. Accepted: November 11, 2023.
UDC:
001.82:[311.21:004(594) 37.014:502.131.1]:311.21 d 10.23947/2334-8496-2023-11 -3-401 -415
Check for updates
Problem Solving and Decision-Making Skills for ESD: A Bibliometric
Analysis
Mujib Ubaidillah1 , Putut Marwoto2 , Wiyanto Wiyanto2 , Bambang Subali2"
1Doctoral Programme of Science Education, Faculty of Mathematics and Natural Science, Universitas Negeri Semarang,
Indonesia, e-mail: [email protected]
2Department of Physics, Faculty of Mathematics and Natural Science, Universitas Negeri Semarang, Indonesia, e-mail: [email protected]: [email protected]: [email protected]
Abstract: Problem-solving and decision-making skills are essential for individuals across various fields. These skills emphasize the importance of preparing a generation capable of solving problems and making informed decisions. Therefore, this study aimed to learn the publication trends related to problem-solving and decision-making skills for ESD (Education for Sustainable Development) from 2013 to 2022 through Bibliometric analysis. In line with the analysis, a VOSviewer software was used to graphically analyze the obtained bibliographic data. A total of 1519 documents were also analytically acquired from the Scopus database. The results showed a fluctuating trend in the number of publications, with the Journal of Chemical Education and Social Sciences being the highest contributor and the most prevalent field of study at 147 and 689 documents, respectively. The United States was also ranked first in the documents emphasizing problem-solving and decision-making skills, at 512 documents. Moreover, the University of Toronto was the most prolific affiliation, contributing the most publications at 17 documents. The representatives from Indonesia were also grouped into two institutions in the global top twenty affiliates, namely (1) the Indonesian University of Education and (2) the State University of Malang. In line with the results, 159 study experts from Indonesia contributed to the analyzed theme, as the top author originated from the United States having 7 documents. The top document excerpts were also published 240 times in the journal Expert Systems with Applications. The trend of the study visualization subsequently produced 9 clusters, problem-solving and decision-making skills, human, psychology, clinical competencies, education, curriculum, support systems, creativity, and content analysis. These results were helpful to relevant experts, regarding the analytical trend in problem-solving and decision-making skills, recommending directions for future analyses.
Keywords: bibliometric analysis, decision-making skills, ESD, problem-solving, VOSviewer.
Introduction
Education for Sustainable Development (ESD) is a new paradigm in the educational field, that provides opportunities to the younger generation regarding contribution to sustainable development (Tripon, 2014). This paradigm was believed to transform the learning mindsets of students, enabling a plan for a better life in the future (Jauhariyah et al., 2021). The ESD curriculum implementation also fosters student's competencies, skills, empathy, responsibility, problem-solving, democratic decision-making, hope, and respect for humanity as well as nature (Laurie et al., 2016). Furthermore, ESD stimulates students to question, analyze, think critically, and make informed decisions (UNESCO, 2012). This paradigm employs an active learning model, which emphasizes academic participation and collaboration.
Based on the aforementioned descriptions, problem-solving and decision-making skills are the methodological competencies required for future success (Erol et al., 2016). This indicated that collaborative decision-making skills were needed in the industrial era 4.0 (Torres, Pimentel and Matias, 2023). The methodological competencies were also selected as essential and relevant characteristics for physics careers (Nasri, Nasri and Talib, 2020), with decision-making skills closely related to the abilities of critical thinking (Molokhina, Pishchik and Fomin, 2021) and communication (Tsai et al., 2020). Therefore, schools need to promote decision-making skills through classroom learning. Collaboration
'Corresponding author: [email protected]
n^x Fr\ I © 2023 by the authors. This article is an open access article distributed under the terms and conditions of the ¡¿^^Kl^H Creative Commons Attribution (CC BY) license (https://creativecommons.Org/licenses/by/4.0/).
among students was also expected to become increasingly important in an environment of complex and uncertain decision-making. This is because the interaction among various stakeholders and knowledge exchange can enhance the processes of making appropriate decisions (Cai et al., 2023). According to a study in various fields, the outcomes obtained were significantly related to decision-making skills in learning mathematics (Pertl, Zamarian and Delazer, 2017), physics (Silvianty, Suhandi and Setiawan, 2019), chemistry (Gao, 2015; Lieber et al., 2022; Wilson and Wilson, 2017), medicine (Delavari et al.,
2022), computers (Carlsson, 2019; Chen and Utama, 2022; Méndez et al., 2020), engineering (Fruchter, Katz and Grey, 2018), business (De Villiers, Hankin and Woodside, 2016), and nursing (Chen et al., 2021; Macduff, Stephen and Taylo, 2016; Saab et al., 2021). Kwok and Hodgson (2017) also stated that decisionmaking model consisted of six steps, namely identifying problems, generating, evaluating, and selecting alternatives, as well as implementing and assessing decisions.
Several study trends in decision-making and problem-solving skills have subsequently been conducted through bibliometrics. This is because bibliometric studies are significantly related to decisionmaking processes in various domains, such as engineering (Cai et al., 2023), universities (Vílchez-Román, Sanguinetti and Mauricio-Salas, 2021), health (Holden, Rosenberg and Barker, 2005), linguistic probability (Liao, Mi and Xu, 2020), business (Costa et al., 2017; Pacheco-Velázquez et al.,
2023), laboratory safety (Abdullah and Abd Aziz, 2021), and artificial intelligence (Di Vaio, Hassan and Alavoine, 2022). These trends are also relevantly associated with problem-solving in the fields of teaching physics (Masitoh et al., 2021) and mathematics education (Chen et al., 2021; Suseelan, Chew and Chin, 2022). A bibliometric study approach is also commonly implemented to emphasize problem-solving and decision-making skills as variables yet to be concurrently investigated by previous experts. This explains that both skills are necessary in sustainable development education across various fields of knowledge. Therefore, this study aimed to identify the trends in publication growth, countries, subject areas, journals, productive authors, university affiliations, highly cited articles, and future experimental opportunities, regarding the concepts of problem-solving and decision-making skills.
The main contribution of the analysis is to provide comprehensive information concerning the trends in publication growth, subject areas, top countries, university affiliations, journals, highly cited articles, productive authors, contributions of Indonesian experts, and future experimental opportunities in the field of problem-solving and decision-making skills. In this case, article authors and journal editors were essentially expected to possess awareness of the disciplinary landscape and a thorough understanding of the analyzed fields, enabling the identification of future experimental opportunities. Policymakers are also capable of recognizing leading countries in the analysis of problem-solving and decision-making skills, which were implemented in the instructional design of several dimensional educators. Therefore, this study aimed to acquire knowledge about problem-solving and decision-making for ESD, through the review and evaluation of the research questions:
1. How has the trend in annual publications, document sources, subject areas, and countries for problem-solving and decision-making skills evolved from 2013 to 2022?
2. How has the distribution of problem-solving and decision-making skills publications evolved based on university affiliation?
3. Who is the most productive writer in the analysis of global problem-solving and decision-making
skills?
4. How has the contribution of Indonesian experts evolved in analyzing problem-solving and decision-making skills?
5. Which paper is most cited and related to problem-solving and decision-making skills?
6. What are the future study opportunities associated with problem-solving and decision-making
skills?
Materials and Methods
A bibliometric analysis approach was employed to draw conclusions based on a collection of bibliographic data, exhibiting the structure, social, and author networks, as well as present analytical interests in a specific experimental area (Ha et al., 2020; Karakus, Ersozlu and Clark, 2019; Suprapto et al., 2021; Zupic and Cater, 2015). The approach also included the statistical analysis of published articles and their citations, to measure relevant impacts (Maditati et al., 2018; Saregar et al., 2022) and analyze gaps or emerging topics (Gao et al., 2021). In addition, bibliographical analysis implemented the relevant information obtained from online databases, enabling scientific study and a global perspective on related interest areas (Secinaro et al., 2020). Figure 1 shows the implementation of the five steps emphasizing
bibliometric method.
Figure 1. Bibliometric methodology
Study design
The study design was executed by formulating research questions, as well as selecting keywords and databases. This indicated the existence of the six questions investigating the following features: i) annual publication trends, document sources, subject areas, and countries,; ii) affiliations,; iii) most productive authors,; iv) contributions of Indonesian experts,; v) highly cited papers,; and vi) future experimental opportunities related to problem-solving and decision-making skills. The design also proved that the implemented search keywords were "Problem-solving decision-making skills", with the Scopus database selected as the bibliometric source.
Data collection
A title search strategy (problem AND solving AND decision AND making AND skills) AND PUBYEAR > 2012 AND PUBYEAR < 2023 were used by the data collection process from the Scopus database. This indicated that 1519 documents for 2013-2022 were acquired from the database, including journals, conference proceedings, review articles, paper chapters, and books. The documents were also downloaded in Comma-Separated Values (CSV) and Research Information System (RIS) formats, to obtain article titles, authors, references, and keywords.
Data analysis
Data analysis was initiated by exporting CSV and RIS information from the Scopus database to Microsoft Excel and VOSviewer. This analysis was conducted to analyze various trends, including publication output characteristics, document sources, country and institutional affiliation distribution, subject category dissemination, top authors and citations, as well as publicized tendencies spanning 2013 to 2022.
Data visualization
The visualization of data was carried out using VOSviewer software (Van Eck and Waltman, 2010, 2012) and Microsoft Excel. By implementing VOSviewer, processed RIS metadata was mapped based on network, overlay, and density visualizations. Meanwhile, the visualization, emphasizing Microsoft Excel, was used to describe the study trends presented as tables and diagrams.
Interpretation
Data visualized by VOSviewer were analyzed and interpreted regarding the number of clusters formed, old and recent studies, and density visualization, which specifically exhibited saturation and future experimental opportunities.
Results
Annual publication trends between 2013-2022
The trend of publication productivity over ten years (2013-2022) was based on the Scopus database, emphasizing relevant fluctuations. In this case, the analytical trends of problem-solving and decisionmaking skills in various disciplines had a 10-years fluctuation. This was evidenced by the document elevations in 2019, 2020, 2021, and 2022 at 196, 172, 197, and 201 publications, respectively. Figure 2 presents the annual publication trends.
Figure 2. Annual publication trends for the period 2013-2022
Main Source Document
The article documents were sourced from journals, proceedings, paper chapters, reviews, and books. Most of these sources focused on problem-solving and decision-making skills from various fields, including education, computer science, engineering, medicine, nursing, public health, psychology, etc. Table 1 shows the twenty most prolific sources for publishing articles related to problem-solving and decision-making skills.
Table 1
Distribution of documents by related sources from 2013-2022
Source SJR index Number of
(Scimagojr 2022) document
Journal of Chemical Education 0.56 (Q2) 147
ASEE Annual Conference and Exposition Conference Proceedings 0(-) 40
Journal of Physics Conference Series 0.18 (-) 18
Advances In Intelligent Systems and Computing O(-) 16
Nurse Education Today 0.95 (Q1) 10
BMC Medical Education 0.91 (Q1) 10
Plos One 0.89(Q1) 9
Proceedings of the European Conference on Games-Based Learning 0(-) 8
International Journal of Environmental Research and Public Health 0.83 (Q2) 8
Sustainability Switzerland 0.37 (Q2) 7
AIP Conference Proceedings 0.16 (-) 7
International Journal of Engineering Education 0.26 (Q2) 6
Frontiers In Psychology 0.89 (Q2) 6
ACM International Conference Proceeding Series 0.21 (-) 6
Journal of Vocational Rehabilitation 0.38 (Q2) 5
Journal of Veterinary Medical Education 0.45 (Q3) 5
Journal of General Internal Medicine 1.81 (Q1) 5
Journal of Engineering Education Transformations 0.21 (Q3) 5
Journal of Dental Education 0.56 (Q2) 5
Thinking Skills and Creativity 1.15 (Q1) 4
Documents based on Subject Area
A maximum of 689 problem-solving and decision-making skills publications were obtained for the 2013-2022 documents emphasizing the subject areas within the field of social sciences. This indicated that "Medicine", "Engineering", 'Computer Science", "Psychology", "Chemistry", "Business, Management, and Accounting", "Health Professions", "Nursing", and "Arts and Humanities" had 364, 246, 237, 162, 153, 94, 79, 78, and 74 publications, respectively. Figure 3 illustrates the document classification emphasizing subject area.
Figure 3. Subject area
Top Publications by Country
Based on country classification, a total of 1519 documents were distributed, with 512, 131, 93, 76, 60, 55, and 42 articles originating from the United States, United Kingdom, Australia, Canada, India, Spain, and Indonesia, respectively. This proved that Indonesia, as a country on the Asian continent, participated in the study trends of America, Europe, and Africa regarding the analysis of problem-solving and decision-making skills. Figure 4 shows the distribution of the top 20 of 102 countries in the publication of problem-solving and decision-making skills.
Figure 4. Top countries publication on problem-solving and decision making skills
University Affiliation
Figure 5 presents the distribution of the top 20 university affiliations emphasizing problem-solving and decision-making skills.
Figure 5. Top affiliation
The distribution of publications was observed from various university affiliates, namely; America, Malaysia, Indonesia, Iran, Australia, England, Africa, etc. In this case, the University of Toronto was the most productive institution with 17 published documents on the Scopus database, accompanied by the Universities of Queensland, Melbourne, and Curtin having 13, 12, and 12 articles, respectively. The Tehran and Indonesian Universities of Medical Sciences and Education also had 11 documents, accompanied by the higher institutions of Pittsburgh, Michigan, and Malang (state) having 10 publications. According to the results, Indonesia contributed two institutions to the top 20 affiliates, namely (1) The Indonesian University of Education, and (2) The State University of Malang.
Based on the Scopus database from 2013 to 2022, the top 10 of 39 Indonesian university affiliates emphasizing the publication of problem-solving and decision-making skills in learning processes were observed. This indicated that the Indonesian University of Education was ranked first, accompanied by the Universities of Malang (state), Bina Nusantara, Mataram, Sebeleas Maret, and Semarang (state). Table 2 presents the top ten affiliates from Indonesia.
Table 2
Top ten affiliations of the University of Indonesia
No Affiliation Number of documents
1 Indonesia University of Education 11
2 State University of Malang 10
3 Bina Nusantara University 4
4 University of Mataram 2
6 Sebelas Maret University 2
7 State University of Semarang 2
8 Kanjuruhan University Malang 2
9 Siliwangi University 1
10 University of Technology 1
Top 10 Authors
According to the evaluation of the top 10 authors regarding the most publications, Shogren, K.A. was ranked first with 7 documents. This was accompanied by Yousefi, M., Suhandi, A., Issa, T., Hooshyar, D., and Ahmad, R.B., which had five (5) documents each. York, S., Shultz, G.V., Raley, S.K., and Horng, S.J. also possessed four publications each. In line with the results, an Indonesian writer, namely Suhandi, A. from the University of Education, analyzed decision-making and problem-solving skills. However, many universities in the country did not publish relevant documents based on the Scopus database. Table 3 illustrates the details for the top 10 authors with the highest number of publications.
Table 3
Ranking of authors with a minimum of four published documents
No Author Affiliation Number of documents Number of citations
1 Shogren, K.A. University of Kansas, Lawrence, United States 7 107
2 Yousefi, M. Universiti Tenaga Nasional, Kajang, Malaysia 5 186
3 Suhandi, A. Indonesia University of Education, Bandung, Indonesia 5 7
4 Issa, T. Curtin University, Australia 5 2
5 Hooshyar, D. University of Malaya, Kuala Lumpur, Malaysia 5 186
6 Ahmad, R.B. University of Malaya, Kuala Lumpur, Malaysia 5 186
7 York, S. University of Nevada, Las Vegas, United States 4 231
8 Shultz, G.V. University of Michigan, Ann Arbor, United States 4 67
9 Raley, S.K. University of Kansas, Lawrence, United States 4 88
10 Horng, S.J. National Taiwan University of Science and Technology, Taiwan 4 177
Contribution of Indonesian Experts Evolved in Problem-solving and Decision-making skills
Based on the Scopus database spanning from 2013 to 2022, the top ten of the 159 authors that published the articles exploring problem-solving and decision-making skills in learning were identified. In this case, Suhandi from the Indonesian University of Education mostly published problem-solving and decision-making themes. However, the entire learning processes emphasizing the themes remained inadequate and limited. Table 4 presents the top 10 authors from Indonesian university affiliates.
Table 4
Top ten author contributions from university affiliates in Indonesia
No Author Affiliation Publication Number of citations
1 Suhandi, A. Indonesia University of Education 5 7
2 Samsudin, A. Indonesia University of Education 3 3
3 Syaodih, E. Indonesia University of Education 2 3
4 Suwono, H. State University of Malang 2 2
5 Ningsih, A. R. Indonesia University of Education 2 3
6 Kaniawati, I. Indonesia University of Education 2 3
7 Hakim, A. University of Mataram 2 35
8 Zarlis, M. North Sumatra University 1 0
9 Zarkasi, A. C University- Nusantara PGRI Kediri 1 0
10 Yuliardi, R. STKIP Muhammadiyah Kuningan 1 1
Document Citation
According to the Scopus database, the top 10 documents with consecutive highest citations were as follows, Liu, Chan and Ran, 2016 (NC=240), Leider and Griffiths, 2019 (NC=220), Ellis et al. 2017
(NC=212), Dineen-Griffin et al. 2019 (NC=144), Orgill et al. (2019; NC=107), Hooshyar et al. 2016 (NC=92), Kazancoglu and Ozkan-Ozen, 2018 (NC=91), Kegler, Stone and Holland, 2017 (NC=89), Opstoel et al. 2020 (NC=83), and Devan et al. 2018 (NC=81). Table 5 illustrates the information on the documents with the most citations.
Table 5
A breakdown of the top 10 documents on Scopus by number of citations
No Authors Year Title Source Number of Citation
1 Liu et al. 2016 Decision-making for the selection of cloud vendor: An improved approach under group decision-making with integrated weights and objective/subjective attributes Expert Systems with Applications 240
2 Lieder & Griffiths 2019 Resource-rational analysis: Understanding human cognition as the optimal use of limited computational resources Behavioral and Brain Sciences 220
3 Ellis et al. 2017 Beyond Risk and Protective Factors: An Adaptation-Based Approach to Resilience Perspectives on Psychological Science 212
4 Dineen-Griffin etal. 2019 Helping patients help themselves: A systematic review of self-management support strategies in primary health care practice PLoS ONE 144
5 Orgill et al. 2019 Introduction to Systems Thinking for the Chemistry Education Community Journal of Chemical Education 107
6 Hooshyar et al. 2016 Applying an online game-based formative assessment in a flowchart-based intelligent tutoring system for improving problem-solving skills Computers and Education 92
7 Kazancoglu & Ozkan-Ozen 2018 Analyzing Workforce 4.0 in the Fourth Industrial Revolution and proposing a road map from an operations management perspective with fuzzy DEMATEL Journal of Enterprise Information Management 91
8 Kegler et al. 2017 Trends in suicide by level of Urbanization -United States, 1999-2015 Morbidity and Mortality Weekly Report 89
9 Opstoel et al. 2020 Personal and social development in physical education and sports: A review study European Physical Education Review 83
10 Devan et al. 2018 What works and does not work in a self-management intervention for people with chronic pain? Qualitative systematic review and meta-synthesis Physical Therapy 81
Mapping Problem Solving and Decision Making with VOSviewer
Co-occurrence Analysis of Keywords
The classification of study subjects related to problem-solving and decision-making skills was interestingly emphasized, as a minimum number of 58,017 keywords occurred, with 743 words meeting the threshold. Network visualization was also categorized into 9 clusters, 42399 links, and 101961 total bond strengths. In this case, the red, green, blue, yellow-green, purple, deep sky blue, orange, Mocca, and light purple clusters prioritized problem-solving and decision-making skills, humans, psychology, clinical competencies, education, curriculum, support systems, creativity, and content analysis through 170, 146, 127, 126, 85, 47, 36, 5, and 1 items, respectively. Regarding the results, the bigger circle led to the high frequency of keywords in the document (Zhang et al., 2022). Figure 6 presents the network visualization display using VOSviewer, emphasizing the most repeated keywords and networks.
Figure 6. Network visualization of co-occurrences with index keywords.
Overlay Visualization
The overlay visualization was symbolized by the colors showing yearly experimental trends, where purple and yellow prioritized years of study and recent analysis, respectively. These recent trends focused on the following terms, namely collaborative learning, gender, e-learning, employee, academic system, social media, formative assessment, project-based education, human experiment, creativity, quality of life, child, educational technology, grounded theory, career, theoretical study, software, games, primary care, anxiety, and machine knowledge. The overlay visualization of co-occurrence is shown in Figure 7.
viitudlaptients
studentfcmedical*
lulatiqo ejfUgr^ni¡coaming
^nf^ ¿fc currjgylurn
•irgeo^
«Ki^llioitphdlllldiy
profcSonal #d *jfi&nurslng
w Wm(liii||mulati(«»
W Re^iî4persQpnel( attitude ^
doi^pr pacijpt ration ^roÊgcffië's v" simula
_ * - ' • ¿»Iftorial emergency health service 'nursi
physician-pa«m«||clations ~ W w t.w ~
5 n J«*»ndotic " , leadership»
patient^«)" ""questionnaire -
Wgftym priority journal * - • ^uantltat® analysis
* TV J^ . quallta,Analysis
• i-T "^psychology human » . .
patient dec^>r#ii«% jj V ■ critical thinking , "»"*"">■
training * electing - «•** „„„»bully
quality of lib evidence based practicc - , , .. c„ .j«^. piWectJa»d learning '»
problem solving • - ifth#iow students ^ .
dg^^e^y^g^cfW***, • deciSI&ffeflng P'oblcm-sofc 18 skills ¿ystemul-ing
dement,' 'V « «I r^ssesvge,,, # .
.- «h„„p» >oun§,?dult bcNavlO''fi««'<lf'isi>wnakîr# , behaviorkresMrcti
comp^Mnon J'.»»» ioffs "V V .* - ..
«MU**« „„^gender
_•••^npil« collaboration
depth m P*chooi
" * • •
chi l^ood
f^VOSviewer
.ina(yri®)i rhemmtry * problem sfl^p^cWstaittivkini
stud^nt-ceft^d^ojnif
Figure 7. Overlay of visualization of co-occurrences with index keywords.
Density Visualization
Density visualization was responsible for the saturation of a topic, where the blue, yellow, and red colors were emphasized yet to be analyzed, rarely studied, and highly evaluated, respectively. This indicated that brighter image colors caused the frequent use of the analyzed term in a relevant experiment. The distribution of keyword density also illustrated the frequency level of study topics (Huang et al.,
2020). For example, problem-solving, human, and decision-making were often analytically implemented, while the rarely analyzed relevant terms focused on problem-based learning, collaboration, sustainability, metacognitive skills, qualitative and quantitative analyses, creativity, life competencies, computational thinking, game-based education, human-computer interaction, childhood, virtual patients, expertise, as well as analytic and hierarchy processes. Figure 8 presents the density visualization of decision-making and problem-solving skills based on keyword index.
students, medical
rsurgeon consensus romputer-assisttSTHOT
clinical competence education, pE
r operating rooms
' professional standard medical education education, nufiSj computer simulation
health personnel attitude
doctor patient relation procedures simulation experiential learning
_J ! , editorial emergency health service nursing student ftfenr relations °
validity pancjemic leadership group process . ------------------anaiyt^i chemistry
^Hnt education
I problem solving/decision
psychometry jQr quantitative antf^
earning teachjpg V ,-. ., _
psychology human sk'" writing 1 h,,,
risionmaking peergioup ^ ^^criticalthinking^^ ^ scaffolds
quality of life evidence based practice ,, , . ....__.project
problem solving technology students
lepression controlled study ma'e decisionmaking problem-solving skills dementia risk assessment risk management «earning systems voune adult . hoh,...... ,„,„,„-h
young adult Savior game decisw n«kino behavioral research ^iemt
employee creativity Sender
behavior game decison making
emotions J . „, ___.____managers product design
daily life activity employee
¡■fBS?^ adolescent_____M *"°™iedge management
parents ch|,d
school child China audsm ",eskHls
brain injury Physiology intelligence^ spon
i perception child, preschool childhood
social skills analytic hierarchy process
Figure 8. Density visualization of topic decision-making and problem-solving based on keywords
index.
Discussions
According to Figure 2, a fluctuating growth for the keywords "problem-solving" and "decisionmaking skills", was observed regarding the publication trend from 2013 to 2022, with the entire productivity remaining relatively positive. This indicated significant document elevations from 2017-2019 and 20202022, underscoring the continuous development of the study topic and providing potential for subsequent exploration of novel aspects. Furthermore, Table 1 showed that the Journal of Chemical Education was the leading source, publishing 147 articles on "problem-solving" and "decision-making skills". These articles extensively explored the interplay between both skills in handson and mindson laboratory activities. For instance, laboratory experiment activities in organic chemistry were commonly problem-based, regarding the following performances: (1) developing students information literacy skills (Shultz and Li, 2016); (2) guiding inquiry lab practices in the quantitative determination of aluminum within deodorant brands (Sedwick et al., 2018); and (3) integrating environmentally friendly chemistry into the curriculum (Aubrecht et al., 2019). Problem-solving was also essential for training students in systemic thinking, decisionmaking, and sustainability.
Based on Figure 3, the publications integrating problem-solving and decision-making skills within the "Social Sciences" discipline totaled 689 documents. This indicated that "Social Sciences" focused on the widest range of disciplines. It also exhibited the significance of the subject matter prioritizing the study, problem-solving, and decision-making skills, which explored the relationship between humans and their social interactions. Furthermore, "medicine" generated 364 publications and was closely related to problem-solving and decision-making skills, which were integral to effective medical practice. This proved that both skills complemented each other and helped doctors in providing high-quality patient care. Medical analysis is also presently integrated with artificial intelligence for problem-solving and decision-making (Secinaro et al., 2021). From the results, "Engineering" subsequently produced 246 publications, which were closely related to both problem-solving and decision-making skills. In this case, an engineer was commonly confronted with the complex tasks requiring technical problem solutions, and rational decisionmaking processes (Kipper et al., 2021). The 'Computer Science' discipline was also observed, with 237
publications emphasizing problem-solving and decision-making skills. These skills were considered an important foundation for developing innovative and effective technical solutions.
In line with the aforementioned descriptions, "psychology" was another discipline that yielded 162 publications, with decision-making and problem-solving skills being relevant integral components (Bar-On, 2010). These skills assisted psychology professionals in handling complex tasks and aiding individuals regarding the achievement of mental and emotional well-being. Moreover, the "Chemistry" field produced 153 documents, with the subject area highly significant as the decision-making and problem-solving skills helping scientists and relevant professionals overcome intricate knowledge and management challenges (Chen et al., 2021). "Business, Management and Accounting" also generated 94 publications, which were pertinent to problem-solving and decision-making skills. These skills emphasized individuals navigating complex business challenges, devising effective strategies, and efficiently managing resources to achieve organizational goals (Proctor, 2010). Another field, "Health Professions", also produced 79 documents, with the significant subject matter focusing on professional-patients relationships, diagnosing diseases, and designing treatment plans, and managing emergency situations. From the results, "Nursing" subsequently produced 78 publications, with the subject matter emphasizing the significance of problem-solving and decision-making skills. This was because the skills prioritized nurse-patient interaction, problem-solving, completing complex tasks, and nursing decisionmaking skills (Ahmady and Shahbazi, 2020). In addition, "Arts and Humanities" provided 74 publications, whose significant subject matter played an essential role in understanding and interpreting relevant works and analyzing cultural contexts.
According to Figure 4, the United States, among 102 countries, was ranked first in document productivity with 512 published articles. This proved that geographic-based information retrieval helped to determine the highest publication distribution in a country (Emmer, 2018; Zupic and Cater, 2015). Figure 5 also showed that the University of Toronto, as a top university affiliate, was ranked first with 17 documents. This was accompanied by the contributions of the Universities of Education and Malang, which represented the Indonesian institutions possessing 11, and 10 documents respectively. Moreover, the contribution of Indonesian experts in the integration of problem-solving and decision-making skills remained low according to the Scopus database. The results also indicated the top 10 contributions from 39 college affiliations within the country, regarding the publication of problem-solving and decision-making skills analyses. From 2013-2022, these publications remained low when perceived as document productivity. Therefore, learning should focus on fostering integrative problem-solving and decision-making skills for implementing education for sustainable development. This was because the skills were crucial in the Industry 4.0 era due to being extensively adopted by workers and managers (Torres, Pimentel and Matias, 2023). Learning including real-world problems also stimulated active student engagement in applying their individual and cognitive knowledge in collaborative problem-solving (Erol et al., 2016).
Based on Table 3, Shogren, K.A. from the University of Kansas, Lawrence, United States, was the most productive author with a total of 7 documents. The author also had the highest number of document citations, totaling 107. Moreover, Yousefi, M., Hooshyar, D., Ahmad, R. B., and Horng, S. J. explored intelligent guidance by using flowcharts to enhance problem-solving and decision-making (Hooshyar et al., 2015). The authors also designed formative assessment with online games in an intelligent tutoring system, to enhance problem-solving and decision-making skills (Hooshyar et al., 2016). In line with the results, York, S. from the University of Nevada, Las Vegas, United States, published 4 documents emphasizing Chemistry Education, with the highest citation of 231. These publications emphasized the systematic thinking approaches in Chemistry Education, to strengthen the conceptual understanding, problem-solving, and decision-making skills related to global sustainability-based issues (Flynn et al., 2019; York and Orgill, 2020). The documents also prioritized the systematic thinking approaches in STEM Education (York et al., 2019) and learning processes applicable to future chemistry educators, experts, and global citizens (Orgill, York and Mackellar, 2019). According to Table 4, the top ten authors from Indonesia were Suhandi, A., Samsudin, A., Syaodih, E., Suwono, H., Kaniawati, I., Hakim, A., Zarlis, M., Zarkasi, A. C., and Yuliardi, R. This confirmed that Suhandi, A. from the Indonesia University of Education was ranked first with 5 published documents, emphasizing physics learning through the implementation of systematic thinking, to enhance problem-solving and decision-making skills. Hakim, A. from the University of Mataram also published 2 documents in the Journal of Chemical Education, with a total of 35 citations. This explained that the chemistry education prioritizing students project laboratories enhanced understanding, problem-solving, and decision-making skills (Hakim, Kadarohman and Syah, 2016, 2020).
In line with the aforementioned descriptions, the citations of a document demonstrated its visibility and importance as a reference in the academic world. This was because the documents with more citations were considered more significant within a specific topic (Zupic and Cater, 2015). The number of references obtained by an article also represented recognition, influence, and popularity within the
scholarly community. Based on Table 5, the authors focused on problem-solving and decision-making skills were exhibited. This indicated that Liu, Chan and Ran (2016) had the highest number of citations at 240, exhibiting the following title:"Decision-making for the selection of cloud vendor: An improved approach under group decision-making with integrated weights and objective/subjective attributes". The second-ranking publication, authored by Lieder and Griffiths in 2019, also had 220 citations with the following title, "Resourcerational analysis: Understanding human cognition as the optimal use of limited computational resources".
According to Figure 6, a keyword network with a minimum occurrence of five terminologies linked to problem-solving and decision-making skills was illustrated. This indicated that the overlay and density visualization analyses were used to identify experimental themes, with future reports emphasizing the VOSviewer outcomes. Several terminologies such as problem-solving, decision-making, and human, were also frequently analyzed. Therefore, the abundance of publications on problem-solving and decisionmaking skills across diverse countries and disciplines prioritized the topic's significance. This emphasized the importance of curricula designation in fostering both skills across various fields. In this case, future studies should employ collaborative approaches, gamification, computational thinking, problem-based learning, e-learning, creativity, and qualitative designs.
Conclusions
In conclusion, this study examined the bibliometric analysis of problem-solving and decision-making skills for ESD (Education for Sustainable Development) to provide insights for future analyses. This showed that the implementation of both skills was necessary across various educational levels and professions. The academic interest in problem-solving and decision-making skills also exhibited fluctuation over the previous decade. Based on the results, the Journal of Chemical Education contributed the highest number of documents, with social sciences being the most prominent subject area. The United States was also the leading contributor of articles, with the University of Toronto being the most productive institution. Moreover, the Universities of Education and Malang (state) were the top-contributing affiliations from Indonesia. Shogren, K.A., from the University of Kansas, Lawrence, United States, was also considered the most productive author with seven documents. In line with the results, Liu, Chan and Ran (2016)was the frequently referenced document published in the journal titled "Expert Systems with Applications". According to the recommendations, numerous potential experimental themes were outlined for future consideration, specifically within the field of education. These themes underscored curriculum design, teaching models, methodologies, assessments, and learning resources, focusing on all levels from primary to higher education. The implications of this research can help related researchers find out trends in problem-solving and decision-making skills for ESD in the educational sector in general. Education for sustainable development should pay special attention to problem-solving and decision-making skills to prepare individuals to face future challenges.
Acknowledgements
The authors are grateful to the PPS-PDD scheme Number 144 /E5/PG.02.00.PL/2023 of the Ministry of Education, Culture, Research, and Technology in 2023, for funding this study.
Conflict of interests
The authors declare no conflict of interest.
Author Contributions
Conceptualization, M.U., P.M., W. and B.S.; Resources, M.U., P.M., W. and B.S; Methodology, M.U., W. and B.S.; Software, M.U., P.M., W. and B.S.; Formal Analysis, M.U., W. and B.S.; Writing -original draft, M.U, P.M., W. and B.S.; Writing - review & editing, M.U., P.W., W. and B.S. All authors have read and agreed to the published version of the manuscript.
References
Abdullah, K. H., & Abd Aziz, F. S. (2021). Mapping of laboratory safety research: a bibliometric review. Malaysian Journal of
Public Health Medicine, 21(1), 303-310. https://doi.org/10.37268/mjphm/vol.21/no.1/art.864 Ahmady, S., & Shahbazi, S. (2020). Impact of social problem-solving training on critical thinking and decision making of nursing students. BMC Nursing, 19(1). https://doi.org/10.1186/s12912-020-00487-x Aubrecht, K. B., Bourgeois, M., Brush, E. J., Mackellar, J., & Wissinger, J. E. (2019). Integrating Green Chemistry in the Curriculum: Building Student Skills in Systems Thinking, Safety, and Sustainability. Journal of Chemical Education, 96(12), 2872-2880. https://doi.org/10.1021/acs.jchemed.9b00354 Bar-On, R. (2010). Emotional intelligence: An integral part of positive psychology. South African Journal of Psychology, 40(1),
54-62. https://doi.org/10.1177/008124631004000106 Cai, Y., Jin, F., Liu, J., Zhou, L., & Tao, Z. (2023). A survey of collaborative decision-making: Bibliometrics, preliminaries,
methodologies, applications and future directions. Engineering Applications of Artificial Intelligence, 122, 106064. Carlsson, C. (2019). Combining ANFIS and Digital Coaching for Good Decisions in Industrial Processes. In K. R.B., B. I., R. M., C. M., & K. V. (Eds.), Advances in Intelligent Systems and Computing (Vol. 1000, pp. 190-200). Springer Verlag. https://doi.org/10.1007/978-3-030-21920-8 18 Chen, L. C., & Utama, D. N. (2022). Decision Support Model for Determining the Best Employee using Fuzzy Logic and Sim-ple
Additive Weighting. Journal of Computer Science, 18(6), 530-539. https://doi.org/10.3844/jcssp.2022.530.539 Chen, L., Kong, D., Zhang, S., & Yang, L. (2021). A quasi-experimental study of specialized training on the clinical decisionmaking skills and social problem-solving abilities of nursing students. Contemporary Nurse, 57(1-2), 4-12. https://doi. org/10.1080/10376178.2021.1912616 Chen, P.-Y., Hwang, G.-J., Yeh, S.-Y., Chen, Y.-T., Chen, T.-W., & Chien, C.-H. (2021). Three decades of game-based learning in science and mathematics education: an integrated bibliometric analysis and systematic review. Journal of Computers in Education, 1-22. https://doi.org/10.1007/s40692-021-00210-y Chen, S.-Y., Tsai, J.-C., Liu, S.-Y., & Chang, C.-Y. (2021). The effect of a scientific board game on improving creative problem
solving skills. Thinking Skills and Creativity, 41. https://doi.org/10.1016/j tsc.2021.100921 Costa, A. L., & Kallick, B. (2009). Habits of mind across the curriculum: Practical and creative strategies for teachers. ASCD. Costa, D. F., de Melo Carvalho, F., de Melo Moreira, B. C., & do Prado, J. W. (2017). Bibliometric analysis on the association between behavioral finance and decision making with cognitive biases such as overconfidence, anchoring effect and confirmation bias. Scientometrics, 111, 1775-1799. https://doi.org/10.1007/s11192-017-2371-5 Devan, H., Hale, L., Hempel, D., Saipe, B., & Perry, M. A. (2018). What works and does not work in a self-management intervention for people with chronic pain? Qualitative systematic review and meta-synthesis. Physical Therapy, 98(5), 381-397. https://doi.org/10.1093/ptj/pzy029 De Villiers, R., Hankin, R., & Woodside, A. G. (2016). Making decisions well and badly: How stakeholders' discussions influence individual executives' decision confidence and competence. Advances in Business Marketing and Purchasing, 24, 87-116. https://doi.org/10.1108/S1069-096420160000024006 Delavari, S., Arabshahi, K. S., Amini, M., Aalaa, M., Pourbairamian, G., Bahoosh, N., Asadi, N., Dalal, B., Kojuri, J., Hamidi, H., & Delavari, S. (2022). The Relationship between Experiences Level and Clinical Decision-Making Skill in Midwifery Students: A Cross-Sectional Study. Medical Journal of the Islamic Republic of Iran, 36(1). https://doi.org/10.47176/ mjiri.36.80
Di Vaio, A., Hassan, R., & Alavoine, C. (2022). Data intelligence and analytics: A bibliometric analysis of human-Artificial intelligence in public sector decision-making effectiveness. Technological Forecasting and Social Change, 174, 121201. https://doi.org/10.1016/j.techfore.2021.121201 Dineen-Griffin, S., Garcia-Cardenas, V., Williams, K., & Benrimoj, S. I. (2019). Helping patients help themselves: A systematic review of self-management support strategies in primary health care practice. PLoS ONE, 14(8). https://doi.org/10.1371/ journal.pone.0220116
Ellis, B. J., Bianchi, J., Griskevicius, V., & Frankenhuis, W. E. (2017). Beyond Risk and Protective Factors: An Adaptation-Based Approach to Resilience. Perspectives on Psychological Science, 12(4), 561-587. https://doi. org/10.1177/1745691617693054 Emmer, A. (2018). Geographies and scientometrics of research on natural hazards. Geosciences, 8(10), 382. https://doi.
org/10.3390/geosciences8100382 Erol, S., Jäger, A., Hold, P., Ott, K., & Sihn, W. (2016). Tangible Industry 4.0: A Scenario-Based Approach to Learning for the
Future of Production. Procedia CIRP, 54, 13-18. https://doi.org/10.1016/j.procir.2016.03.162 Flynn, A. B., Orgill, M., Ho, F. M., York, S., Matlin, S. A., Constable, D. J. C., & Mahaffy, P. G. (2019). Future Directions for Systems Thinking in Chemistry Education: Putting the Pieces Together. Journal of Chemical Education, 96(12), 3000-3005. https://doi.org/10.1021/acs.jchemed.9b00637 Fruchter, R., Katz, G., & Grey, F. (2018). From technical to tactical, and strategic BIM coordination in support of model-based decision making. In Transforming Engineering Education: Innovative Computer-Mediated Learning Technologies (pp. 173-196). American Society of Civil Engineers (ASCE). https://doi.org/10.1061/9780784414866.ch06 Gao, P., Meng, F., Mata, M. N., Martins, J. M., Iqbal, S., Correia, A. B., Dantas, R. M., Waheed, A., Xavier Rita, J., & Farrukh, M. (2021). Trends and future research in electronic marketing: A bibliometric analysis of twenty years. Journal of Theoretical and Applied Electronic Commerce Research, 16(5), 1667-1679. https://doi.org/10.3390/jtaer16050094 Gao, R. (2015). Incorporating students' self-designed, research-based analytical chemistry projects into the instrumentation
curriculum. Journal of Chemical Education, 92(3), 444-449. https://doi.org/10.1021/ed500502w Ha, C. T., Thao, T. T. P., Trung, N. T., Huong, L. T. T., Dinh, N. Van, & Trung, T. (2020). A Bibliometric Review of Research on STEM Education in ASEAN: Science Mapping the Literature in Scopus Database, 2000 to 2019. Eurasia Journal of Mathematics, Science and Technology Education, 16(10). https://doi.org/10.29333/ejmste/8500 Hakim, A., Jufri, A. W., & Mutmainnah, P. A. (2020). Understanding the Uniqueness of Artocarpus Flavonoids: Isolation and Structure Elucidation of Cycloartocarpin from the Roots of Artocarpus altilis. Journal of Chemical Education, 97(11),
4133-4136. https://doi.orq/10.1021/acs.jchemed.0c00199 Hakim, A., Kadarohman, A., & Syah, Y. M. (2016). Making a Natural Product Chemistry Course Meaningful with a Mini Project
Laboratory. Journal of Chemical Education, 93(1), 193-196. https://doi.orq/10.1021/ed500930s Holden, G., Rosenberg, G., & Barker, K. (2005). Bibliometrics: A potential decision making aid in hiring, reappointment, tenure
and promotion decisions. Social Work in Health Care, 41(3-4), 67-92. https://doi.org/10.1300/J010v41n03 03 Hooshyar, D., Ahmad, R. B., Yousefi, M., Fathi, M., Horng, S.-J., & Lim, H. (2016). Applying an online game-based formative assessment in a flowchart-based intelligent tutoring system for improving problem-solving skills. Computers and Education, 94, 18-36. https://doi.org/10.1016/j.compedu.2015.10.013 Hooshyar, D., Ahmad, R. B., Yousefi, M., Yusop, F. D., & Horng, S.-J. (2015). A flowchart-based intelligent tutoring system for improving problem-solving skills of novice programmers. Journal of Computer Assisted Learning, 31(4), 345-361. https://doi.org/10.1111/jcal.12099 Huang, C., Yang, C., Wang, S., Wu, W., Su, J., & Liang, C. (2020). Evolution of topics in education research: a systematic review using bibliometric analysis. Educational Review, 72(3), 281-297. https://doi.org/10.1080/00131911.2019.1566 212
Jauhariyah, M. N. R., Prahani, B. K., Syahidi, K., Deta, U. A., Lestari, N. A., & Hariyono, E. (2021). ESD for physics: How to infuse education for sustainable development (ESD) to the physics curricula? Journal of Physics: Conference Series, 1747(1). https://doi.org/10.1088/1742-6596/1747/1/012032 Karakus, M., Ersozlu, A., & Clark, A. C. (2019). Augmented reality research in education: A bibliometric study. Eurasia Journal
of Mathematics, Science and Technology Education, 15(10). https://doi.org/10.29333/ejmste/103904 Kazancoglu, Y., & Ozkan-Ozen, Y. D. (2018). Analyzing Workforce 4.0 in the Fourth Industrial Revolution and proposing a road map from operations management perspective with fuzzy DEMATEL. Journal of Enterprise Information Management, 31(6), 891-907. https://doi.org/10.1108/JEIM-01-2017-0015 Kegler, S. R., Stone, D. M., & Holland, K. M. (2017). Trends in suicide by level of Urbanization - United states, 1999-2015.
Morbidity and Mortality Weekly Report, 66(10), 270-273. https://doi.org/10.15585/mmwr.mm6610a2 Kipper, L. M., lepsen, S., Dal Forno, A. J., Frozza, R., Furstenau, L., Agnes, J., & Cossul, D. (2021). Scientific mapping to identify competencies required by industry 4.0. Technology in Society, 64, 101454. https://doi.org/10.1016/j. techsoc.2020.101454
Kwok, P. M. H., & Hodgson, P. (2017). Practicing decision making: Using flowcharts with twenty-first-century marketers. International Journal of Pedagogy and Curriculum, 24(2), 1-14. https://doi.org/10.18848/2327-7963/CGP/v24i02/1-14 Laurie, R., Nonoyama-Tarumi, Y., Mckeown, R., & Hopkins, C. (2016). Contributions of Education for Sustainable Development (ESD) to Quality Education: A Synthesis of Research. Journal of Education for Sustainable Development, 10(2), 226-242. https://doi.org/10.1177/0973408216661442 Liao, H., Mi, X., & Xu, Z. (2020). A survey of decision-making methods with probabilistic linguistic information: bibliometrics, preliminaries, methodologies, applications and future directions. Fuzzy Optimization and Decision Making, 19, 81-134. https://doi.org/10.1007/s10700-019-09309-5 Lieber, L. S., Ibraj, K., Caspari-Gnann, I., & Graulich, N. (2022). Students' Individual Needs Matter: A Training to Adaptively Address Students' Argumentation Skills in Organic Chemistry. Journal of Chemical Education, 99(7), 2754-2761. https://doi.org/10.1021/acs.jchemed.2c00213 Lieder, F., & Griffiths, T. L. (2019). Resource-rational analysis: Understanding human cognition as the optimal use of limited
computational resources. Behavioral and brain sciences, 43, e1. https://doi.org/10.1017/S0140525X1900061X Liu, S., Chan, F. T. S., & Ran, W. (2016). Decision making for the selection of cloud vendor: An improved approach under group decision-making with integrated weights and objective/subjective attributes. Expert Systems with Applications, 55, 37-47. https://doi.org/10.1016/j.eswa.2016.01.059 Macduff, C., Stephen, A., & Taylor, R. (2016). Decision precision or holistic heuristic?: Insights on on-site selection of student
nurses and midwives. Nurse Education in Practice, 16(1), 40-46. https://doi.org/10.1016/j.nepr.2015.06.008 Maditati, D. R., Munim, Z. H., Schramm, H. J., & Kummer, S. (2018). A review of green supply chain management: From bibliometric analysis to a conceptual framework and future research directions. Resources, Conservation and Recycling, 139, 150-162. https://doi.org/10.1016/j.resconrec.2018.08.004 Masitoh, P. N. A., Latifah, S., Saregar, A., Aziz, A., & Jamaluddin, W. (2021). Bibliometric analysis of physics problem solving.
Journal of Physics: Conference Series, 1796(1), 12009. https://doi.org/10.1088/1742-6596/1796/1/012009 Méndez, B. E. P., Calleros, J. M. G., Garciá, C. A. R., & Garciá, J. G. (2020). Fuzzy Models for Implementation of the Deci-sion-Making Module in Networked Didactic Prototypes. Computacion y Sistemas, 24(2), 539-551. https://doi.org/10.13053/ CyS-24-2-3378
Molokhina, G. A., Pishchik, V. I., & Fomin, A. (2021). Features of critical thinking of individual entrepreneurs with different levels of self-regulation. International Journal of Cognitive Research in Science, Engineering and Education, 9(2), 253-264. https://doi.org/10.23947/2334-8496-2021-9-2-253-264 Nasri, N. M., Nasri, N., & Talib, M. A. A. (2020). Physics teachers' perceptions on sustainable physics education. Journal of
Baltic Science Education, 19(4), 569-582. https://doi.org/10.33225/jbse/20.19.569 Opstoel, K., Chapelle, L., Prins, F. J., De Meester, A., Haerens, L., van Tartwijk, J., & De Martelaer, K. (2020). Personal and social development in physical education and sports: A review study. European Physical Education Review, 26(4), 797-813. https://doi.org/10.1177/1356336X19882054 Orgill, M., York, S., & Mackellar, J. (2019). Introduction to Systems Thinking for the Chemistry Education Community. Journal
of Chemical Education, 96(12), 2720-2729. https://doi.org/10.1021/acs.jchemed.9b00169 Pacheco-Velázquez, E. A., Vázquez-Parra, J. C., Cruz-Sandoval, M., Salinas-Navarro, D. E., & Carlos-Arroyo, M. (2023). Business Decision-Making and Complex Thinking: A Bibliometric Study. Administrative Sciences, 13(3), 80. https://doi. org/10.3390/admsci13030080
Pertl, M.-T., Zamarian, L., & Delazer, M. (2017). Reasoning and mathematical skills contribute to normatively superior decision making under risk: evidence from the game of dice task. Cognitive Processing, 18(3), 249-260. https://doi.org/10.1007/ s10339-017-0813-x
Proctor, T. (2010). Creative problem solving for managers: developing skills for decision making and innovation. Routledge.
Saab, M. M., Hegarty, J., Murphy, D., & Landers, M. (2021). Incorporating virtual reality in nurse education: a qualitative study
of nursing students' perspectives. Nurse Education Today, 105, 105045. https://doi.Org/10.1016/j.nedt.2021.105045 Saregar, A., Sunyono, Haenilah, E. Y., Hariri, H., Putra, F. G., Diani, R., Misbah, & Umam, R. (2022). Natural Disaster Education in School: A Bibliometric Analysis with a Detailed Future Insight Overview. International Journal of Educational Methodology, 8(4), 743-757. https://doi.org/10.12973/ijem.8.4.743 Secinaro, S., Brescia, V., Calandra, D., & Biancone, P. (2020). Employing bibliometric analysis to identify suitable business
models for electric cars. Journal of Cleaner Production, 264, 121503. https://doi.Org/10.1016/j.jclepro.2020.121503 Secinaro, S., Calandra, D., Secinaro, A., Muthurangu, V., & Biancone, P. (2021). The role of artificial intelligence in healthcare: a structured literature review. bMc medical informatics and decision making, 21, 1-23. https://doi.org/10.1186/s12911-021-01488-9
Sedwick, V., Leal, A., Turner, D., & Bakarr Kanu, A. (2018). Quantitative Determination of Aluminum in Deodorant Brands: A Guided Inquiry Learning Experience in Quantitative Analysis Laboratory. Journal of Chemical Education, 95(3), 451-455. https://doi.org/10.1021/acs.jchemed.7b00336 Shultz, G. V, & Li, Y. (2016). Student Development of Information Literacy Skills during Problem-Based Organic Chemistry Laboratory Experiments. Journal of Chemical Education, 93(3), 413-422. https://doi.org/10.1021/acs.jchemed.5b00523 Silvianty, A., Suhandi, A., & Setiawan, W. (2019). Video supported critical thinking test in the kinetic theory of gases: Validity and reliability. In P. I., A. R.R., S. null, S. null, N. A.B.D., & A. A.G. (Eds.), Journal of Physics: Conference Series (Vol. 1157, Issue 3). Institute of Physics Publishing. https://doi.org/10.1088/1742-6596/1157/3/032052 Suprapto, N., Sukarmin, S., Puspitawati, R. P., Erman, E., Savitri, D., Ku, C. H., & Mubarok, H. (2021). Research trend on TPACK through bibliometric analysis (2015-2019). International Journal of Evaluation and Research in Education, 10(4), 1375-1385. https://doi.org/10.11591/IJERE.V10I4.22062 Suseelan, M., Chew, C. M., & Chin, H. (2022). Research on mathematics problem solving in elementary education conducted from 1969 to 2021: A bibliometric review. International Journal of Education in Mathematics, Science, and Technology (IJEMST), 10(4), 1003-1029. https://doi.org/10.46328/ijemst.2198 Torres, D., Pimentel, C., & Matias, J. C. (2023). Characterization of Tasks and Skills of Workers, Middle and Top Managers in
the Industry 4.0 Context. Sustainability, 15(8), 6981. https://doi.org/10.3390/su15086981 Tripon, A. (2014). Innovative Technology for Sustainable Development of Human Resource Using Non-formal and Informal
Education. Procedia Technology, 12, 598-603. https://doi.org/10.1016/j.protcy.2013.12.535 Tsai, M. N., Liao, Y. F., Chang, Y. L., & Chen, H. C. (2020). A brainstorming flipped classroom approach for improving students' learning performance, motivation, teacher-student interaction and creativity in a civics education class. Thinking Skills and Creativity, 38(July), 100747. https://doi.org/10.1016/j.tsc.2020.100747 UNESCO. (2012). Education for Sustainable Development in Action. In United Nations Educational, Scientific and Cultural Organization.
Van Eck, N. J., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping. Scien-
tometrics, 84(2), 523-538. https://doi.org/10.1007/s11192-009-0146-3 Van Eck, N. J., & Waltman, L. (2012). Manual for VOSviewer version 1.5.2. Leiden Univeristy, September, 1-28. http://www.
vosviewer.com/documentation/Manual VOSviewer 1.5.4.pdf Vílchez-Román, C., Sanguinetti, S., & Mauricio-Salas, M. (2021). Applied bibliometrics and information visualization for decisionmaking processes in higher education institutions. Library Hi Tech, 39(1), 263-283. https://doi.org/10.1108/LHT-10-2019-0209
Wilson, M. V, & Wilson, E. (2017). Authentic performance in the instrumental analysis laboratory: Building a visible spectrophotometer prototype. Journal of Chemical Education, 94(1), 44-51. https://doi.org/10.1021/acs.jchemed.6b00515 York, S., Lavi, R., Dori, Y. J., & Orgill, M. (2019). Applications of Systems Thinking in STEM Education. Journal of Chemical
Education, 96(12), 2742-2751. https://doi.org/10.1021/acs.jchemed.9b00261 York, S., & Orgill, M. (2020). ChEMIST Table: A Tool for Designing or Modifying Instruction for a Systems Thinking Approach in Chemistry Education. Journal of Chemical Education, 97(8), 2114-2129. https://doi.org/10.1021/acs.jchemed.0c00382 Zhang, L., Carter, R. A., Qian, X., Yang, S., Rujimora, J., & Wen, S. (2022). Academia's responses to crisis: A bibliometric analysis of literature on online learning in higher education during COVID-19. British Journal of Educational Technology, 53(3), 620-646. https://doi.org/10.1111/bjet.13191 Zupic, I., & Cater, T. (2015). Bibliometric Methods in Management and Organization. Organizational Research Methods, 18(3), 429-472. https://doi.org/10.1177/1094428114562629
