CC BY
13Central Asian Journal of Medical Hypotheses and Ethics
2020; Vol 1 (2)
© 2020 by the authors. This work is licensed under Creative Commons Attribution 4.0 International License https://creativecommons.org/licenses/by/4.0/
eISSN: 2708-9800 https://doi.org/10.47316/cajmhe.2020.12.03
RESEARCH METHODOLOGY
OPINION
REGISTERING AND REPORTING SYSTEMATIC REVIEWS
Received: Dec. 11, 2020 Accepted: Dec. 28, 2020
Pallavi Patro1 https://orcid.org/0000-0002-7645-5883
Durga Prasanna Misra2* https://orcid.org/0000-0002-5035-7396
1School of Telemedicine, Sanjay Gandhi Postgraduate Institute of Medical Sciences (SGPGIMS),
Lucknow, India
2Department of Clinical Immunology and Rheumatology, Sanjay Gandhi Postgraduate Institute of Medical Sciences (SGPGIMS), Lucknow, India
Corresponding author: Durga Prasanna Misra, Assistant Professor, Department of Clinical Immunology and Rheumatology, Sanjay Gandhi Postgraduate Institute of Medical Sciences (SGPGIMS), Lucknow-226014, India. Twitter handle: @DurgaPrasannaM1; E-mail: durgapmisra@gmail.com
Abstract
Systematic reviews are considered as the highest rung in the ladder of evidence-based medicine. They are bound by a pre-defined structure and requirement for extensive literature searches, when compared with the more liberal format of narrative reviews. Systematic review protocols should ideally be pre-registered to avoid duplication or redundancy. After defining clear review question(s), thorough literature searches form the basis of systematic reviews. Presentation of results should be qualitative or quantitative (meta-analysis) if the data is homogenous enough to permit pooling across multiple studies. Quality of individual studies by Cochrane risk of bias 2 tool for interventional studies and other suitable scales for observational studies, as well as appropriate assessment of publication bias are recommended. Certainty of outcomes should be assessed by the GRADE profiler. Finally, systematic reviews should conclude with recommendations for future research, based on their findings.
Keywords: Systematic review, Bibliography as topic, Meta-analysis, Bias
How to cite: Patro P, Misra DP. Registering and reporting systematic reviews. Cent Asian J Med Hypotheses Ethics 2020;1 (2):122—126. https://doi.org/10.47316/caimhe.2020.12.03
SYSTEMATIC
REVIEWS
AND
EVIDENCE-BASED MEDICINE
Evidence-based medicine has advanced in leaps and bounds over the past three decades. The appraisal of evidence relies on the type of studies on which such evidence is based. Systematic reviews have been considered the highest form of evidence base [1]. They rely on either qualitatively or quantitatively summating information across multiple different studies. Systematic reviews could be performed for both interventional studies and observational studies. Moreover, systematic reviews can make indirect comparisons across different studies using techniques such as network meta-
analyses [2]. The aim of this article is to overview the principles underlying systematic reviews.
SYSTEMATIC VERSUS NARRATIVE REVIEWS
A systematic review aims to systematically search the literature and present it, while critically analysing study quality of individual studies and their outcomes. A systematic literature review minimizes bias in the selection of articles for systematic review. Many systematic reviewers also recommend searches across clinical trial databases and abstracts of major conferences in the specialty in the recent past to identify
studies that might be relevant yet unpublished. As we shall discuss subsequently, systematic reviews are bound by a structure. Since they collate information across studies, systematic reviews require a lot of effort from authors. as a consequence, systematic reviews are generally considered as original research work [2].
Narrative reviews, on the other hand, are more liberal in their structure. They often incorporate the opinion of the author group on a particular matter, something which is limited in a systematic review. Although not considered mandatory, a systematic literature search is highly recommended before embarking on writing a narrative review. This is meant to reduce the extent of bias in the selection of articles, thereby, enhances quality of narrative reviews. Searches for unpublished studies are not mandatory for a narrative review [2, 3].
SYSTEMATIC REVIEW GUIDELINES
Guidelines for conducting systematic reviews have been prescribed by the Cochrane collaboration in the form of the Cochrane Handbook of Systematic Reviews for Interventions [4]. Guidelines for reporting systematic reviews are the Preferred Reporting Items for Systematic Reviews and Meta-analyses [5]. In addition, Gasparyan et al have proposed guidelines for systematic and comprehensive search strategies which could be consulted by authors of systematic reviews [3].
PROTOCOL REGISTRATION
The first step in conducting a systematic review is to design and publish a protocol. It is important to critically search the literature to identify pre-existing systematic reviews on the topic. Unless the proposed systematic review incorporates recently published, critical information in the field, it risks redundancy and might be a futile exercise. After identifying the need for the said systematic review, it is essential to pre-plan the protocol, detailing all steps as proposed, and then to publish this. Systematic review protocols can be pre-registered on databases such as the prospective review of systematic reviews (PROSPERO) or the Cochrane database of systematic reviews (CDSR). The CDSR is reserved for systematic reviews conducted under the aegis of the Cochrane collaboration. Pre-registering systematic reviews helps to avoid redundancy. Also, external reviewers are likely to confirm the final systematic review report for concordance with the pre-planned protocol. Major deviations from the protocol, unless justified adequately, might risk introducing bias into a systematic review [2].
REVIEW QUESTION
The systematic review process begins with the definition of one or more review questions. These should be well-defined, based on the PICO (patients, interventions, comparators, outcome) format. Inclusion and exclusion criteria for studies should be clearly defined and adhered to. However, at the time of protocol design, such criteria should be carefully designed so as to be able to identify at least a few relevant studies. It is preferable to pilot the review process during protocol design to ensure that relevant studies are identified, to avoid wasting effort otherwise on a systematic review without a meaningful number of studies to collate together [2].
SEARCH STRATEGY
A thorough, diverse literature search is the backbone of a systematic review. This should encompass multiple bibliographic databases. For example, systematic reviews in the field of biomedicine would be considered incomplete if they do not search at least one of Scopus or Web of Science, which are the two largest databases. Furthermore, search terms or key words used during such literature searches should be presented in detail in the search strategy, including the number of results at each stage, narrowing down to the final retrieved literature searches [6]. The preferred reporting standards for systematic reviews and meta-analyses (PRISMA) have presented guidelines for presenting search strategies, detailing identification of studies, screening and reasons for exclusion, identifying eligible studies based on detailed assessment of screened full texts and reasons for exclusion, with the final number of included studies. A limitation of the PRISMA guidelines is that they mandate reporting search results from at least a single database. This might not be comprehensive enough, and the authors suggest that such search guidelines should be supplemented by those proposed by Gasparyan et al for conducting systematic literature searches across multiple databases [3].
Apart from database searches, it is also considered essential to search for unpublished yet relevant literature, such as clinical trial databases while conducting systematic reviews of interventional studies. A search on the World Health Organization (WHO) International Clinical Trials Registry Portal (ICTRP) is considered adequate, since multiple regional databases, including that of the National Institutes of Health (clinicaltrials.gov), feed into the ICTRP. Similarly, it is recommended to hand-search conference abstracts of major international and regional conferences in the specialty, since they might help identify relevant studies
which have not yet been published. Limiting such conference abstract searches to the past three or five years helps avoid including studies which might not have been published eventually due to their lack of quality or relevance. It is recommended to duplicate literature searches using at least two investigators [2].
EXTRACTING INFORMATION
Relevant information to fulfil the review objectives from selected studies should be extracted on to pre-designed proformas. This step should also be repeated in duplicate at least to avoid missing relevant information due to random errors during information retrieval [7].
STUDY QUALITY AND RISK OF BIAS
The quality of individual studies should be assessed using the Cochrane risk of bias 2 tool (RoB 2) for interventional studies. Studies are scored based on risk of bias in randomization, adherence, determination of the outcome, the potential for selective data reporting and missing data. Studies can have low risk of bias, some concern about risk of bias or high risk of bias, based on algorithms presented in the tool. Of note, there is no category of "no risk of bias"[8, 9]. For observational studies, various scoring systems such as the Newcastle-Ottawa scale are used [10, 11].
PUBLICATION BIAS
Negative studies might go unpublished, hence a thorough assessment of evidence requires critical analysis of publication bias. This is generally feasible when there are at least ten studies available in a particular area. This can be performed by using funnel plots and statistical tests such as the Egger test or Begg's test [11, 12].
CERTAINTY OF EVIDENCE
For outcomes observed across multiple studies for a single comparison, the GRADE profiler helps analyse the certainty of evidence for a particular outcome, based on consistency of results, risk of bias, precision of estimates, and the directness or indirectness of evidence. The GRADE profiler assigns a score of very low, low, moderate or high certainty of evidence based on the inputs provided [13, 14].
REPORTING RESULTS
The summary of findings table helps present the characteristics of individual studies included in a systematic review [13, 15]. The information so retrieved may be presented in a quantitative format (meta-analysis) pooling information across studies using
summary measures. Meta-analysis should be performed if the identified studies are homogenous enough to permit pooling of data. Such pooled estimates should be analysed for the heterogeneity of estimates. Random effects meta-analyses should pool data from heterogenous studies, otherwise fixed-effects metaanalysis can be used. Pooled results are presented in the form of forest plots [11, 16].
META-ANALYSES
Recent guidelines recognise systematic reviews incorporating synthesis without meta-analyses (SWIM) [17]. As discussed above, meta-analyses are not essential for systematic reviews and should not be conducted inappropriately, for example, when pooling diverse outcomes from methodologically distinct studies [2].
CONCLUSION
Systematic reviews are the cornerstone of EBM, considered as the highest level of evidence when conducted across multiple randomized trials for a particular intervention. Most guidelines or recommendations for disease management are preceded by systematic reviews, which are also then published separately. Systematic reviews should conclude with suitable recommendations for further research based on their review findings. It is important to avoid redundancy in systematic reviews by pre-publishing protocols. Recent instances of automatically generated but redundant, duplicated systematic reviews of little actual relevance to science have been noted, often generated commercially and sold to authors. Such practices have questionable ethics and should be avoided [18, 19]. Inexperienced authors willing to conduct systematic reviews themselves are highly recommended to undergo specialist training for the same, as provided in courses conducted by the Cochrane collaboration and other organizations.
FUNDING
None
AUTHOR CONTRIBUTIONS
Substantial contributions to the conception or design of the work; and the acquisition, analysis, or interpretation of data for the work - PP, DPM. Drafting the work - PP, DPM. Revising it critically for important intellectual content - PP, DPM. Final approval of the version to be published - PP, DPM. Agreement to be accountable for all aspects of the work in ensuring that questions related
to the accuracy or integrity of any part of the work are appropriately investigated and resolved. - PP, DPM.
CONFLICTS OF INTEREST
The authors have no potential conflicts of interest to disclose.
DISCLAIMER
No part of the manuscript has been submitted simultaneously or published elsewhere.
REFERENCES
1. Levels of evidence - Oxford Centre for Evidence Based Medicine [Internet] http://www.cebm.net/oxford-centre-evidence-based-medicine-levels-evidence-march-2009/ [accessed on 21 November 2017].
2. Misra DP, Agarwal V. Systematic Reviews: challenges for their justification, related comprehensive searches, and implications. J Korean Med Sci. 2018;33(12):9.
3. Gasparyan AY, Ayvazyan L, Blackmore H, Kitas GD. Writing a narrative biomedical review: considerations for authors, peer reviewers, and editors. Rheumatol Int 2011 ;31 (11 ):1409-1417.
4. Front Matter. Cochrane handbook for systematic reviews of interventions 2019. p. i-xxviii.
5. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and metaanalyses: the PRISMA statement. PLoS Med 2009;6(7):e1000097-e.
6. Searching for and selecting studies. Cochrane handbook for systematic reviews of interventions 2019. p. 67-107.
7. Collecting data. Cochrane handbook for systematic reviews of interventions 2019. p. 109-141.
8. Assessing risk of bias in a randomized trial. Cochrane handbook for systematic reviews of interventions 2019. p. 205-228.
9. Sterne JAC, Savovic J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019;366:l4898.
10. Newcastle-Ottawa scale. http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp [Accessed on Dec 10, 2020].
11. Misra DP, Zimba O, Gasparyan AY. Statistical data presentation: a primer for rheumatology researchers. Rheumatol Int 2020 doi: 10.1007/s00296-020-04740-z. Online ahead of print.
12. Lin L, Chu H. Quantifying publication bias in meta-analysis. Biometrics 2018;74(3):785-794.
13. Completing 'Summary of findings' tables and grading the certainty of the evidence. Cochrane handbook for systematic reviews of interventions 2019. p. 375-402.
14. GRADEpro GDT: GRADEpro Guideline Development Tool [Software]. McMaster University, 2020 (developed by Evidence Prime, Inc.). Available from gradepro.org.
15. Summarizing study characteristics and preparing for synthesis. Cochrane handbook for systematic reviews of interventions 2019. p. 229-240.
16. Analysing data and undertaking meta-analyses. Cochrane handbook for systematic reviews of interventions 2019. p. 241-284.
17. Campbell M, McKenzie JE, Sowden A, et al. Synthesis without meta-analysis (SWiM) in systematic reviews: reporting guideline. BMJ 2020;368:l6890.
18. Misra DP, Ravindran V, Agarwal V. Integrity of authorship and peer review practices: challenges and opportunities for improvement. J Korean Med Sci 2018;33(46):14.
19. Misra DP, Agarwal V. Integrity of clinical research conduct, reporting, publishing, and post-publication promotion in rheumatology. Clin Rheumatol 2020;39:1049-1060.
ЖYЙЕЛIК ШОЛУДЫ Т1РКЕУ ЖЭНЕ ЕСЕП БЕРУ
TY^HieMe
ЖYйелi шолулар дэлелдi медицинанын ен жогары децгейi болып саналады. Олар гылыми шолуларды орындайтын либералды форматтардан ерекшеленетiн/ эдебиетгi iздеуге арналган алдын-ала жасалган цурылыммен жэне талаптармен сипатталады. Ен дурысы, цайталануды болдырмау мацсатында жYйелiк шолу хаттамаларын алдын-ала таркеу цажет. Шолу сурацтары нацты аныцталганнан кейiн/ зерттеудiн негiзiн цурайтын эдебиеттердi муцият iздеу жYргiзiледi. Нэтижелердi усыну сапалы немесе сандыц болуы керек (метаанализ), егер де мэлiметтер бiрнеше зерттеулердi бiрiктiруге жеткiлiктi бiртектi болса. Жеке зерттеулердщ сапасын багалау Yшiн интервенциялыц зерттеулерге арналган ЖYЙелi ;ателштер ;ayni Кокранов ^¥ралы 2 немесе бацылаушы зерттеулерге арналган басца да тшсп шкалаларды цолдану усынылады, сонымен ;атар, басылымдардын вздершщ ЖYЙелi ;ателштершщ багалануы керек. Нэтижелердiн дэлдiгiн GRADE профайлерi багалауы керек. Сонымен, жYйелi шолулар цорытындыларга негiзделген болашац зерттеулерге арналган усыныстармен аяцталуы керек.
TYйiндi свздер: ЖYйелiк шолу, тацырып ретшде библиография, метаанализ, бейiмдiлiк Дэйексвз Yшiн: Патро П., Мисра Д.П. ЖYйелiк шолуды тiркеу жэне есеп беру. Медициналыц гипотеза мен этиканын Орта Азиялыц журналы. 2020; 1(2): 122-126. https://doi.Org/10.47316/caimhe.2020.1.2.03
РЕГИСТРАЦИЯ И ОТЧЕТНОСТЬ СИСТЕМАТИЧЕСКИХ ОБЗОРОВ
Резюме
Систематические обзоры считаются высшей ступенью доказательной медицины. Они характеризуются заранее установленными структурой и требованиями к поиску литературы, чем отличаются от более либеральных форматов выполнения научных обзоров. В идеале протоколы систематических обзоров должны быть предварительно зарегистрированы во избежание дублирования. После четкого определения вопросов обзора, выполняется тщательный поиск литературы, который формирует основу исследования. Представление результатов должно быть качественным или количественным (метаанализ), если данные достаточно однородны для объединения нескольких исследований. Для оценки качества отдельных исследований рекомендуется использовать Кокрановский инструмент риска систематических ошибок 2 для интервенционных исследований или другие подходящие шкалы для наблюдательных исследований, а также необходима оценка самих публикаций на наличие систематических ошибок. Точность результатов должен оценивать профайлер GRADE. Наконец, систематические обзоры должны завершаться рекомендациями для будущих исследований, составленными на основании полученных выводов.
Ключевые слова: Систематический обзор, библиография как тема, метаанализ, предвзятость Для цитирования: Патро П., Мисра Д.П. Регистрация и отчетность систематических обзоров. Центральноазиатский журнал медицинских гипотез и этики. 2020; 1(2): 122-126. https://doi.Org/10.47316/caimhe.2020.1.2.03
