CC BY-ND
12
National Research University Higher School of Economics Journal of Language & Education Volume 5, Issue 2, 2019
Raitskaya, L., Tikhonova, E. (2019). Scoping Reviews: What is in a Name? Journal of Language and Education, 5(2), 4-9. doi: https://doi.org/10.17323/jle.2019.9689
Editorial
Correspondence concerning this article should be addressed to Lilia Raitskaya, MGIMO University, 76 Pr. Vernadskogo, Moscow, Russia, 119454. E-mail: [email protected]
Elena Tikhonova
National Research University Higher School of Economics; Peoples' Friendship University of Russia (RUDN University)
Correspondence concerning this article should be addressed to Elena Tikhonova, National Research University Higher School of Economics, 26 Shabolovka, Moscow, Russian Federation, 119049.
E-mail: [email protected]
The editorial dwells upon scoping or otherwise called mapping reviews that have recently come to the fore. Starting to appear from the early 2000s, scoping reviews initially came out in medicine and biosciences. The present-day unprecedented boost in the scoping review quantity is spurred by a general thrust for structured analysis and synthesis of scientific information across fields and disciplines. The authors aim to overview the methodology of scoping reviews with regard to their prospects for social sciences and humanities.
Keywords: scoping review, mapping review, systematic review, umbrella review, meta-analysis, review of literature, review typology, critical analysis, theoretical review
Reviews of literature stand alone among numerous scientific publications. They are produced by expert researchers who structure and evaluate the knowledge across research fields. On average, reviews tend to attract more attention than articles. The tradition of reviews dates back to the 18th century. But even today novel types of reviews turn up. As some authors note "new approaches to review production have emerged that facilitate efficiency, particularly in the context of review production" ..., including "... online software platforms, the use of text mining and machine learning" (Elliott et al., 2017).
Starting with the early 2000s, scoping reviews came into being. In the Scopus database, the number of such reviews has been rising since 2015 unprecedentedly fast. The bulk of the reviews come out in medicine and health professions, considerably outpacing other fields. In the 2010s, social sciences and humanities started to turn to some comparatively new forms of reviews, including scoping reviews. Their share in the general pool of scoping reviews is rising, though not as fast to outpace their popularity in medicine.
Typology of Reviews
Taxonomy of reviews as independent and stand-alone published studies may count on diverse criteria: their goals; their scope and volume; search and selection strategies; data extraction and quality appraisal techniques; depth of the analysis; methodology frameworks and so on. M.Grant and A.Booth (2009) offered a 14-type classification. One of the most recent typologies was developed by Paré et al. (2015). They focussed on recurrent first-order constructs, or otherwise defined as dimensions to identify each of the eight types of reviews in their typology. Given the theme of this editorial, we have simplified the typology, boiling down their three types of systematic reviews to meta-analyses and systematic reviews (as compared with their original types covering meta-analyses, qualitative systematic reviews, and realist reviews).
This article is published under the Creative Commons Attribution 4.0 International License.
Figure 1. Typology of Reviews.
Note: Adapted from Information & Management, NL, 52(2), 183-199. Copyright 2014 by Elsevier B.V.
To decide between diverse reviews, researchers should consider all considerations and limitations. Each type of reviews has some striking features, both strong and weak points in synthesizing previously published studies. True to their name, narrative reviews overview the literature relating to a field of study in a narrative form. They do not aim to select sources in any comprehensive or profound manner. Narrative reviews "present verbal descriptions of past studies ..." and "... are of heuristic value" (King & He, 2005). A non-systematic approach to synthesis coupled with vague methodology constitute major drawbacks of narrative reviews amid a general thrust for structured synthesis of scientific information.
Narrative Reviews
Descriptive Reviews
Scoping Reviews
Meta-Analyses
Systematic Reviews
Critical Reviews
Theoretical Reviews
Umbrella Reviews
Scope I Depth
✓
// ✓ //
/ ✓
/ ✓ /
Aim
Focus
Methodology
field description some sources
/ field trends literature
/ field scopc literature at large, gaps
/ // research questions empirical findings
✓ / research questions empirical findings
✓ critical assessment literature on a broad topic
/ conceptual framework novel conceptualisations
✓ narrow research questions systematic reviews
unstructured s c arc h
structured search & analysis
inclusion & exclusion criteria
statistical analysis
qualitative analysis, subjective methods
acceptability criteria
taxonomies and frameworks
qualitative analysis, subjective methods
Legend
/
//
///
to an extent
to a medium extent
to a greater extent
Figure 2. Key Characteristics of Review Types.
Descriptive reviews seek to "collect, codify, and analyze numeric data ... found in the extant literature" (King & He, 2005). Being rather broad in scope, they aim to identify the trends in the field. "Each study included in a descriptive review is treated as a unit of analysis, and the published literature as a whole provides a database" (Paré et al., 2015). The prevailing methods applied are content and frequency analyses. Sometimes such reviews are labelled as 'the state of the art reviews'.
"True to their name, scoping reviews are an ideal tool to determine the scope or coverage of a body of literature on a given topic" (Munn, 2018). This type is relatively novel, but more and more researchers are choosing to conduct such a study as it enables to filter lots of titles and find gaps in the field.
The term 'meta-analysis' was coined in 1976 by G.V. Glass who defined it as "the analysis of analyses" (Glass, 1976). This form of reviews was rather rare before the 1970s. It is "much less judgmental and subjective than other literature review methods" (King & He, 2005). The primary specific feature of meta-analysis is its focus on research data instead of conclusions of the studies under reviewing. Meta-analyses are more popular in fields typical of evidence-based research.
Systematic reviews "have been considered as the pillar on which evidence-based healthcare rests" (Munn et al., 2018). This type of reviews is thoroughly developed. In an attempt to strengthen "journal research reporting through the use of reporting guidelines" (McLeroy et al., 2012), the scholarly community offers guidelines for producing meta-analysis and systematic reviews. In medicine, systematic reviews strictly follow methodology and has a complex typology. The systematic review methodology may prompt the selection process to get less biased (Knobloch, Yoon & Vogt, 2011). Effectiveness reviews, experimental reviews, risk reviews, prognostic reviews, psychometric reviews and others are found among systematic reviews. Systematic reviews aim "to synthesize and appraise all relevant high-quality research in an effort to answer a specific research or clinical question" (Vrabel, 2015). In fields beyond medicine, this type of reviews is gaining popularity, based on somewhat simplified methodologies.
Some review types turn up quite rarely. They are more narrow-focussed. Their goals are rather clear-cut. In critical reviews an idea of critical approach is at the core. They "include a degree of analysis and conceptual innovation" (Grant & Booth, 2009). As such, they are narrative or conceptual and may result in building up a hypothesis.
Theoretical reviews, or reviews of concepts are conducted to fill in the gaps by analysing the theoretical framework and building novel conceptualisation. They "use various structured approaches such as classification systems, taxonomies and frameworks to organize prior research effectively, examine their interrelationships" (Paré et al., 2015).
On top of all, the recent trend towards conducting review studies across all fields has resulted in reviews of reviews called 'umbrella reviews', often referred to as meta-reviews emerging now and then. "Umbrella reviews can be considered a broader term" (Biondi-Zoccai, 2016), covering other similar meta-reviews, including overviews of reviews and meta-epidemiological reviews. This review is defined as a review of compelling evidence from multiple reviews into one accessible and usable document (Biondi-Zoccai, 2016).
Scoping Reviews on the Rise
To estimate the popularity of scoping reviews across all fields over the years of their existence, we extracted data from the Scopus database. After searching Scopus for 'scoping review' as the keyword, we found 3,093 document results with the earliest title published in 2005. The data for 2019 was incomplete as the search brought the documents as of June 29, 2019. One document was indexed for 2020 (see Graph 1).
The annual distribution of scoping reviews is uneven and has been skyrocketing since 2015-2016. The rising popularity reflects the spread of scoping reviews into new fields and disciplines.
The search brought various document types. Of 3,093, reviews accounted for 1,834 documents, articles amounted to 1,062. The articles might be divided into two categories. Their overwhelming majority were mislabelled as 'articles', being true scoping reviews. The remaining few in the category were articles on scoping review methodology or techniques.
The search brought various document types. Of 3,093, reviews accounted for 1,834 documents, articles amounted to 1,062. The articles might be divided into two categories. Their overwhelming majority were mislabelled as 'articles', being true scoping reviews. The remaining few in the category were articles on scoping review methodology or techniques.
Source: Scopus Database. Graph 1. Scoping Reviews in Scopus Database Between 2005 and 2019 (as of June 29, 2019).
2,160 documents in the primary search related to medicine. Nursing, social sciences, health professions, and psychology brought 499, 497, 306, and 256 documents respectively. Documents labelled as 'Arts and Humanities' totalled 71. Subjects of most scoping reviews in social sciences and humanities were substantially associated with medicine or health professions.
We also conducted an electronic search for 'scoping study', resulting in 254 documents (as of June 29, 2019), with 1-2 documents dating back to the 1970s and 1980s. The breakdown by publication types shows 159 articles (but most of them are labelled as articles, being true reviews), 44 conference papers, and 36 reviews.
Scoping reviews and studies have a great unrealised potential in social sciences and humanities. Following their framework-based methodology, they might efficiently map emerging fields and revise the established areas to determine their foci trends and detect the existing gaps in the knowledge.
Scoping Review Methodology
Like all scholarly publications in medicine, a scoping review is based on well-wrought methodology framework. It was originally developed by H. Arksey & L. O'Malley (Arksey & O'Malley, 2005), who approached the five-stage framework in a transparent and well-structured manner. All stages fit in a selection and valuation algorithm depicted in Fig.3.
The methodology authors highlight that Stage 1 focusses on refining research questions. Too vague or wide definitions may result in "an unmanageably large numbers of references" (Arksey & O'Malley, 2005).
After having filtered the literature with the inclusion and exclusion criteria in mind, Stage 2 brings the studies relevant to the research aim and questions. The stage implies searching databases, reference lists, existing networks or any other sources. In some reviews, researchers find it useful to hand-search some journals and conference proceedings. In sorting out titles, researchers often redefine search words to provide for a comprehensive title and source coverage. In addition to the search strategy, at Stage 2 eligibility criteria are fixed (Halas et al., 2015). Simultaneously, all duplicates are removed. One should also note that terminology should be clarified at the outset to be sure that no misuse of terms may lead to a plethora of irrelevant papers in the searches. Sometimes foreign (non-Anglophone) authors stick to different terms even when they publish in English, using calques or equivalents for the terms from their mother tongues.
Figure 3. Scoping Review Methodology Framework.
Note: Adapted from International Journal of Social Research Methodology, UK, 8(1), 19-32. Copyright 2005 by Taylor & Francis.
During Stage 3, researchers thoroughly assess the selected studies (their abstracts and texts, if necessary) against the inclusion and exclusion criteria to make up a flow chart, reporting the selection results. "Study selection is not linear, but rather an iterative process that involves searching the literature, refining the search strategy, and reviewing articles for study inclusion." (Colquhoun et al., 2014).
Stage 4 embraces categorisation of the titles based on the extracted data. The final stage summarizes the findings relating to the research question(s). At Stage 5, a field scope as well as foci trends and knowledge gaps are identified to reveal if any further in-depth study is needed.
Concluding Remarks
Social sciences and humanities will have to go a long way to come to understanding of review-based knowledge synthesis. With scholarly information rising in an uncontrolled manner, various synthesised approaches to vast volumes of literature would easily provide insights into the state of the art in any field of knowledge. Thus, researchers would get an overview and identify gaps in no time. Reviews are not only synthesised data and information at large, but thorough studies with assessments and analyses. They are "a focal point in assessing the epistemological progress of any field" (Pahlevan-Sharif, Mura & Wijesinghe, 2019). Skills to conduct review studies are essential for any researcher. An overview of literature incorporated in articles of all types is similar to reviews. It is advisable that researchers follow some review methodology frameworks to meet the best practices in overviewing literature.
The present JLE editorial is a glimpse of the review domain for our potential authors and readers to join. New reviews on educational and language issues are sure to enrich the JLE content and help researchers in their further studies.
References
Arksey, H., &0'Malley,L.(2005).Scopingstudies:Towardsamethodological framework.InternationalJournalofSocial
Research Methodology: Theory and Practice, 8(1), 19-32. doi: https://doi.org/10.1080/1364557032000119616 Biondi-Zoccai, G. (Ed.) (2016). Umbrella reviews. Evidence synthesis with overviews of reviews and meta-epidemiologic
studies. Switzerland: Springer. doi: https://doi.org/10.1007/978-3-319-25655-9 Colquhoun, H. L., Levac, D., O'Brien, K. K., Straus, S., Tricco, A. C., Perrier, L., . . . Moher, D. (2014). Scoping reviews: Time for clarity in definition, methods, and reporting. Journal of Clinical Epidemiology, 67(12), 12911294. doi: https://doi.org/10.1016/j.jclinepi.2014.03.013 Elliott, J. H., Synnot, A., Turner, T., Simmonds, M., Akl, E. A., McDonald, S., . . . Pearson, L. (2017). Living systematic
review: 1. Introduction - he why, what, when, and how. Journal of Clinical Epidemiology, 91, 23-30. doi: https:// doi.org/10.1016/j.jclinepi.2017.08.010 Halas, G., Schultz, A. S. H., Rothney, J., Goertzen, L., Wener, P., & Katz, A. (2015). A scoping review protocol to map the research foci trends in tobacco control over the last decade. BMJ Open, 5(1). doi: https://doi. org/10.1136/bmjopen-2014-006643 Glass, G.V. (1976). Primary, secondary, and meta-analysis of research. Review of Research in Education, 5(10), 351-379.
Grant, M., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies.
Health Information and Libraries Journal, 26(2), 91-108. doi: https://doi.org/10.1111/j.1471-1842.2009.00848.x King, W. R., & He, J. (2005). Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems, 16, Article 32, 665-686. doi: https://doi.org/10.17705/1CAIS.01632 Available at: https://aisel.aisnet.org/cais/vol16/iss1/32 Knobloch, K., Yoon, U., & Vogt, P. M. (2011). Preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement and publication bias. Journal of Cranio-Maxillofacial Surgery, 39(2), 91-92. doi: https:// doi.org/10.1016/j.jcms.2010.11.001 McLeroy, K. R., Northridge, M. E., Balcazar, H., Greenberg, M. R., & Landers, S. J. (2012). Reporting guidelines and the American journal of public health 's adoption of preferred reporting items for systematic reviews and meta-analyses. American Journal of Public Health, 102(5), 780-784. doi: https://doi.org/10.2105/ AJPH.2011.300630
Munn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18(1). doi: https://doi.org/10.1186/s12874-018-0611-x Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018). What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Medical Research Methodology, 18(1). doi: https://doi.org/10.1186/s12874-017-0468-4 Pahlevan-Sharif, S., Mura, P., & Wijesinghe, S.N.R. (2019). A systematic review of systematic reviews in tourism.
Journal of Hospitality and Tourism Management, 39, 158-165. doi: https://doi.org/10.10Wj.jhtm.2019.04.001 Paré, G., Trudel, M., Jaana, M., & Kitsiou, S. (2015). Synthesizing information systems knowledge: A typology of literature reviews. Information and Management, 52(2), 183-199. doi: https://doi.org/10.10Wj.im.2014.08.008 Vrabel, M. (2015). Preferred reporting items for systematic reviews and meta-analyses. Oncology Nursing Forum, 42(5), 552-554. doi: https://doi.org/10.1188/15.0NF.552-554
