BASIC ASPECTS OF META-ANALYSIS. PART 1 Текст научной статьи по специальности «Фундаментальная медицина»

Original articles / Оригинальные статьи https://doi.org/10.47093/2218-7332.2023.14.1.4-14

D) Check for updates

BY 4.0

Basic aspects of meta-analysis. Part 1

Alexander Yu. Suvorov, Irina V. Latushkina®, Kseniya А. Gulyaeva, Nikolay M. Bulanov,

Maria Yu. Nadinskaia, Alexey A. Zaikin

Sechenov First Moscow State Medical University (Sechenov University) 8/2, Trubetskaya str., Moscow, 119991, Russia

Abstract

Meta-analysis is one of the concepts of scientific methodology, and is a frequent but optional component of systematic reviews of empirical research. It joins the results of several scientific studies and tests one or more interrelated scientific hypotheses using quantitative (statistical) methods. This analysis can either use primary data from the original studies or published (secondary) results of studies dealing with the same problem. Meta-analysis is used to obtain an estimate of the magnitude of an unknown effect, and compare the results of different studies, identifying patterns or other relationships in them, as well as possible sources of disagreement. Meta-analyses are the highest level of credibility within evidence-based medicine (EBM), so meta-analysis results are considered as the most reliable source of evidence. Understanding all the procedures of a meta-analysis will allow researchers to analyze the results of such studies correctly, as well as formulate tasks when conducting meta-analyses on their own. In this article the reader will be introduced to key concepts such as weighted effects, heterogeneity, the different types of statistical models used, and how to work with some of the types of plots produced in meta-analyses.

Keywords: effect size; fixed effects model; random effects model; heterogeneity; sensitivity analysis; randomized controlled trial; cohort study MeSH terms:

META-ANALYSIS AS TOPIC

For citation: Suvorov A.Yu., Latushkina I.V., Gulyaeva K.A., Bulanov N.M., Nadinskaia M.Yu., Zaikin A.A. Basic aspects of meta-analysis. Part 1. Sechenov Medical Journal. 2023; 14(1): 4-14. https://doi.org/10.47093/2218-7332.2023.14 14-14

CONTACT INFORMATION:

Irina V. Latushkina, junior researcher, Centre for Analysis of Complex Systems, Sechenov First Moscow State Medical University (Sechenov University)

Address: 8/2, Trubetskaya str., Moscow, 119991, Russia

Tel.: +7 (916) 126-12-85

E-mail: latushkina_i_v@staff.sechenov.ru

Conflict of interests. The authors declare that there is no conflict of interests.

Financial support. This article was supported by the Academic leadership program Priority 2030 proposed by Sechenov First Moscow State Medical University (Sechenov University)

Received: 22.12.2022 Accepted: 09.02.2023 Date of publication: 30.03.2023

УДК [616.98:578.834.1]-092

Базовые аспекты мета-анализа. Часть 1

А.Ю. Суворов, И.В. Латушкинан, К.А. Гуляева, Н.М. Буланов, М.Ю. Надинская,

А.А. Заикин

ФГАОУ ВО «Первый Московский государственный медицинский университет им. И.М. Сеченова»

Минздрава России (Сеченовский Университет) ул. Трубецкая, д. 8, стр. 2, г. Москва, 119991, Россия

Аннотация

Мета-анализ - одно из понятий научной методологии. Он является частым, но не обязательным компонентом систематического обзора эмпирических исследований. Для проведения мета-анализа объединяются результаты нескольких научных исследований и осуществляется проверка одной или нескольких взаимосвязанных научных гипотез при помощи количественных (статистических) методов. Для такого анализа можно использовать либо первичные данные оригинальных исследований, либо обобщенные опубликованные (вторичные) результаты исследований, посвященные одной проблеме. Мета-анализ используется для получения оценки величины неизвестного эффекта, а также для сравнения результатов различных исследований, выявляет в них закономерности или другие взаимосвязи, а также возможные источники разногласий. Мета-анализы занимают высшую ступень достоверности в концепции доказательной медицины, поэтому их результаты считаются самым надежным источником доказательств. Понимание всех этапов проведения мета-анализа позволит научным сотрудникам грамотно анализировать результаты таких исследований, а также формулировать задачи при самостоятельном проведении мета-анализов. В настоящей статье читатель познакомится с такими ключевыми понятиями мета-анализа, как взвешенные эффекты, гетерогенность, различные типы используемых статистических моделей, а также научится работать с некоторыми видами графиков, получаемых в мета-анализах.

Ключевые слова: размер эффекта; модель с фиксированными эффектами; модель со случайными эффектами; гетерогенность; анализ чувствительности; рандомизированное контролируемое исследование; когортное исследование

Рубрики MeSH:

МЕТА-АНАЛИЗ КАК ТЕМА

Для цитирования: Суворов А.Ю., Латушкина И.В., Гуляева К.А., Буланов Н.М., Надинская М.Ю., Заикин А.А. Базовые аспекты мета-анализа. Часть 1. Сеченовский вестник. 2023; 14(1): 4-14. https://doi.org/10.47093/2218-7332.2023.14.1.4-14

КОНТАКТНАЯ ИНФОРМАЦИЯ:

Латушкина Ирина Викторовна, младший научный сотрудник Центра анализа сложных систем ФГАОУ ВО «Первый МГМУ

им. И.М. Сеченова» Минздрава России (Сеченовский Университет)

Адрес: ул. Трубецкая, д. 8, стр. 2, г. Москва, 119991, Россия

Тел.: +7 (916) 126-12-85

E-mail: latushkina_i_v@stafF.sechenov.ru

Конфликт интересов. Авторы заявляют об отсутствии конфликта интересов.

Финансирование. Статья подготовлена при поддержке программы стратегического академического лидерства «Приори-тет-2030» ФГАОУ ВО «Первый Московский государственный медицинский университет им. И.М. Сеченова» Минздрава России (Сеченовский Университет).

Поступила: 22.12.2022 Принята: 09.02.2023 Дата печати: 30.03.2023

List of abbreviations

CI - confidence interval

NRT - non-randomized trial RCT - randomized controlled trial

Many original studies have similar research objectives but often the research teams, patients, research protocol and time intervals are different. The results of such studies can be diverse and contradictory, which hampers clinical decision-making. Evidence-based medicine has led to the development of tools for combining the results of numerous studies that may differ in certain areas [1]. We get (a) invaluable data whose effect can be traced in any groups (or, conversely, only in specific ones) (b) information about the variability of the effect when testing hypotheses in different populations. Many similar studies are replications of one large experiment, and, accordingly, a larger number of replications increases the power and the degree of confidence in the results.

There are several basic tools to evaluate the combined results of the similar studies: • Systematic reviews. Systematic reviews include all studies to be found that meet certain strict inclusion criteria. The inclusion criteria are designed to meet the set of standards required for planning and conducting research, as well as standard operating procedures and results (for example, studies on arterial hypertension, assessing blood pressure according to cardiology international guidelines;

studies of specific surgery, conducted on the international association guidelines). • Meta-analyses. Meta-analysis not only summarizes the results of the set of studies, but also quantifies them. In fact, we need to know not only that the drug/ intervention has an effect, but also to assess its effect size and the range of its variability. In this review, we will mention the basic aspects of conducting meta-analyses and tell you what to do after searching the literature and doing most of the work for a systematic review.

INCLUDING STUDIES IN META-ANALYSIS PRISMA guidelines

A detailed description of the criteria and the process of literary search is not the subject of this review, but it is important to note that any creation of a systematic review consists of a set of items. These items are combined into a standard scheme called the PRISMA flow diagram (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), by the name of the relevant recommendations [2] (Fig. 1).

In reality, meta-analysis begins after the inclusion is completed and the studies from which the data extraction will be performed are available.

^ 'A

& и

2 g

О й

Identification in science research databases (PubMed, Scopus, WoS, elibrary, and others) / Идентификация по базам данных научных публикаций (PubMed, Scopus, WoS, elibrary и других)

Exclusion / Исключение

Screening based on title, abstract / Скрининг на основании названия, введения n = ...

Other topics, patient populations / Иные тематики, популяции пациентов n = ...

Suitable for the chosen purpose / Подходящие в рамках выбранной цели

Other primary or concomitant disease, inability to identify the cohort of interest / Иное основное или сопутствующее заболевание, невозможность выделить интересующую когорту n = ...

Available full-text articles / Доступные полнотекстовые статьи n = ...

FIG. 1. PRISMA flow diagram template, adapted from M.J. Page et al. [2]

РИС. 1. Шаблон потоковой диаграммы PRISMA, адаптирована из M.J. Page и соавт. [2]

n —

n —

The main guidelines for the initial and further steps of practical interest to the reader are:

• Cochrane Handbook for Systematic Reviews of Interventions1 - this guide is posted online and describes in detail almost all literary search issues, meta-analysis, basic skills in mathematical statistics required for such work;

• PRISMA guidelines, which we discussed above [2]. These guidelines will help to maintain high

standards of writing systematic reviews and reduce the number of possible errors and inaccuracies that may complicate the continuation of the work. Meta-analyses are widely used by drug companies, as well as other commercial entities, and therefore observance of guidelines standards is very carefully checked by reviewers both when reviewing and when publishing articles. Strict observance of regulation and guidelines is the key to a successful publication.

EVALUATION OF POTENTIAL BIASES IN PUBLICATIONS

After all suitable publications have been collected for inclusion in a systematic review or meta-analysis,

it is necessary to evaluate them in terms of potential biases.

Unfortunately, there are many sources of potential bias, so special tools have been developed to allow researchers to conduct a potential evaluation of publications. Such tools are called risk of bias plots.

These plots can be built for studies with different design types, primarily for randomized controlled trials (RCT) (Fig. 2) and non-randomized trials (NRT) [3, 4] (Fig. 3). The plots below are called "traffic lights". The basic idea is that researchers conducting a meta-analysis with the inclusion of RCTs (Fig. 2) review each study separately and assess the risks associated with the following five domains:

• randomization;

• deviations from intended interventions;

• missing data;

• measurement of outcome;

• selection of reported results.

The assessed risk is "high", "some concerns" and "low".

If NRTs are included in the meta-analysis, we assess the risks associated with the following seven domains:

Risk of bias domains / Домены риска смещений

D1 / Д1 D2 / Д2 D3 / Д3 D4 / Д4 D5 / Д5 Overall / Общая оценка

Study 1 / Исследование 1 ® ® ® ® ® ®

Study 2 / Исследование 2 ® ® ® G ®

Study 3 / Исследование 3 ® О О ® ® О

Study 4 / Исследование 4 ® ® <X) ® О

Study 5 / Исследование 5 О ® e ® ® ®

Judgement / Оценка риска

® high / высокий some concerns / средний (+) low / низкий

FIG. 2. Traffic light plot for randomized controlled trials, adapted from L.A. McGuinness et al. [5]

РИС. 2. Диаграмма светофор для рандомизированных контролируемых исследований, адаптирована из L.A. McGuinness и соавт. [5]

Note: risk of bias associated with the domains: D1 - randomization; D2 - deviations from intended interventions; D3 - missing data; D4 - measurement of outcome; D5 - selection of reported results.

Примечание: риск смещения, ассоциированный с доменами: Д1 - рандомизацией; Д2 - вмешательством; Д3 - пропущенными данными; Д4 -оценкой конечной точки; Д5 - представлением результатов.

1 Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). Cochrane, 2021. Available from https://training.cochrane.org/handbook/current

Risk of bias domains / Домены риска смещений

D1 / Д1 D2 / Д2 D3 / Д3 D4 / Д4 D5 / Д5 D6 / Д6 D7 / Д7 Overall / Общая оценка

Study 1 / Исследование 1 о ® Ф (D ® ® ® О

Study 2 / Исследование 2 ® ® ® ® ® ® ® <X)

Study 3 / Исследование 3 ® ® ® Ф ® ® CD

Study 4 / Исследование 4 (D О ® Ф CD CD Ф О

Study 5 / Исследование 5 Ф о ® ® ® ® CD G)

Judgement / °ценка риска

(j^ critical / очень высокий (х) serious / высокий C-^ moderate / умеренный (+) low / низкий

FIG. 3. Traffic light plot for cohort studies, adapted from L.A. McGuinness et al. [5]

РИС. 3. Диаграмма светофор для когортных исследований, адаптирована из L.A. McGuinness и соавт. [5]

Note: risk of bias associated with the domains: D1 - confounding; D2 - selection of participants; D3 - classification of interventions; D4 - deviations from intended interventions; D5 - missing data; D6 - measurement of outcome; D7 - selection of reported results.

Примечание: риск смещения, ассоциированный с доменами: Д1 - конфаундинг (влияние вмешивающихся факторов); Д2 - отбором и включением пациентов; Д3 - вмешательством; Д4 - расхождением с протоколом; Д5 - пропущенными данными; Д6 - оценкой конечной точки; Д7 - представлением результатов.

confounding; selection of participants; classification of interventions; deviations from intended interventions; missing data; measurement of outcome; selection of reported results.

The assessed risk is interpreted as "critical", "serious", "moderate" and "low".

Specified tools enable critical approaches to the results obtained in the meta-analysis and consider studies with a high risk of bias as less reliable. A detailed description of the capabilities of this tool is provided on a specialized website2.

EFFECT SIZE IN META-ANALYSIS

The results of studies combined in a meta-analysis are measured by an identical endpoint. This measurement is an effect that has been achieved, or an observed effect (abbreviated as 0 or 0k for each of the k studies). The definitions of the effect, effect size and effect size measuring are described in our publication on statistical hypotheses testing [6].

There are two main concepts that allow us to describe and measure the effect of several studies. Both

concepts relate to certain statistical models, with fixed and random effects, respectively.

Fixed (common) effects model

This is a model in which the studies included in the meta-analysis are very similar to each other in terms of design, number of patients, methodology, evaluation of results and other items, and their results or effects 0123 k are considered a single sample from one general population of all possible similar studies.

The probability distribution of such a population has an expected value (the mean weighted by the probabilities of possible values), which represents a certain true effect size 0. Each study is a part of the population, and several studies randomly taken from such distribution (meta-analysis) is an ordinary sample, respectively.

The observed effect of each study k will differ from the true one by the error:

0 = 0k +

We believe that among several studies, the studies with the smallest sample error e are the most accurate (Fig. 4).

https://www.riskofbias.info/ (access date 01.11.2022).

2

The general population of studies / Генеральная совокупность исследований

/Т\

Individual studies / Отдельные исследования

n = 15

n = 90

n =1000 -

Number of patients in the study / Число пациентов в исследовании

True effect size / Истинный размер эффекта

Observed effects / Наблюдаемые эффекты

FIG. 4. Schematic representation of true and observed effects in studies using a fixed effects model

РИС. 4. Схематичное представление истинного и наблюдаемых эффектов в исследованиях при использовании модели с фиксированными эффектами

Note: ек - sampling error; grey line - true effect size; colored straight lines - observed effects.

Примечание: ek - величина выборочной ошибки; серая линия - истинный размер эффекта; цветные прямые линии - наблюдаемые эффекты.

A sample from several studies {1,2,3,...k} must have some central tendency or expected value indicating the true effect size. Thus, using the definition of expected value as a mean weighted, we get:

„ w,e, + we + w0, + ••• + w.e.

r\ _ 11 22 33 k k

w, + w + w. + ... + w,

12 3 k

where

• 0 - weighted effect size for k studies resulting from meta-analysis;

• 0k - the observed effect of the study k;

• wh - study weight k.

However, it is still unclear how to achieve the weight of each study. We know that the observed effect obtained in study k is a point estimate. Study k includes a certain set of patients, n. The standard error is a measure of the variability of the effect 0k and is calculated as:

in

where

• a - the standard deviation of the effect 0 in the

k

study k;

• n - number of patients in the study k.

In a fixed effects model, the inverse variance method is one of the ways to calculate weights:

w, = -

1

where

• s2 - the square of the standard error of the effect 0k

in the study k;

• wk - weight of the study k.

Thus, in a model with fixed effects, the weight of any study is inversely related to the inverse effect error in the study and directly related to the number of patients included in the study. The model implies that only the number of patients can affect the weight of the study.

This concept seems to be overly simplified, because in the real world, there are a huge number of different factors apart from the sample size.

Random effects model

If we combine different designs (RCTs, cohorts, etc.) in a meta-analysis studies conducted in different periods of time, in different countries, in hospitals with different standards of medical care, with intervention protocols according to different clinical guidelines, we will need a model that will take into account not only differences in sample size, but also the above-mentioned factors.

In this type of model, the effect for each study included in the meta-analysis is a sample from its own set of effect sizes and differs from the expected value of its own set by the ek (Fig. 5).

If we included k studies in the meta-analysis, there are k samples from k different general populations. At the same time, k populations have their own distribution with the expected value represented by

Distribution of several general populations / Распределение нескольких генеральных совокупностей

/ТУ

I

I

I

I

I

I

I

I

I

I

General populations / Генеральные совокупности

Л

Individual studies / Отдельные исследования

I t I I I 1 I I I I 1 I I I 1 I 1 1 I I

True effect size / Истинный размер эффекта

Observed effects / Наблюдаемые эффекты

Observed effects

Наблюдаемые эффекты

t

FIG. 5. Schematic representation of true and observed effects in studies using a random effects model

РИС. 5. Схематичное представление истинного и наблюдаемых эффектов в исследованиях при использовании модели со случайными эффектами

Note: - error of each general population; ek - sampling error; grey line - true effect size; colored dotted lines - observed effects in general populations; colored straight lines - in individual studies.

Примечание: £k - величина ошибки каждой генеральной совокупности; ek - величина выборочной ошибки; серая линия - истинный размер эффекта; цветные пунктирные линии - наблюдаемые эффекты в генеральной совокупности; цветные прямые линии - в отдельных исследованиях.

weighted effect size, and each general population differs from the expected value of its distribution by £k. This overall distribution has a point estimate, which corresponds to the meta-analysis weighted effect, 0, and variance t2.

Thus, the point estimate of the effect in each study among k differs from the weighted effect the following way:

0 = 0k + S + ^

The parameter combines the differences that are not related to the sampling error.

The model that uses this logic is called the random effects model. The individual studies weights in such a model are calculated the following way:

1

W = ■ 2 I 2 •

k s 2 + T2 •

where

• s2 - the square of the standard error of the effect 0k in the study k;

• wk - study weight k;

• t2 - the variance of several general populations. And the weighted effect is calculated in the same

way as for the fixed effects model:

w 0+ w 0+ w 0+ + w 0

ri-jV/j 2 2 3 3 k k

w , + wn + w„ + ... + w,

12 3 k

where

• 0 - weighted effect size for k studies resulting from meta-analysis;

• 0k - the observed effect of the study k;

• wk - study weight k.

Various mathematical approaches are used to calculate parameter t2, most often DerSimonian-Laird, Restricted Maximum Likelihood, Maximum Likelihood, Paule-Mandel estimators but there are others [7-9]. The choice of a specific method depends on the type of measurement of endpoint and on the specific situation, therefore it requires consultation with a biostatistician.

HETEROGENEITY ASSESSMENT

We discovered that the studies included in the meta-analysis can vary significantly, moreover, depending on these differences, one or another analysis model is chosen. Is there any measure that can assess the degree of differences? Can we somehow explain the degree of differences and is it necessary? How to determine which studies are more different from others? The concept of heterogeneity can answer all these questions.

Heterogeneity depends on many things, with the most common being:

• incorrect selection of studies for meta-analysis;

• the presence of overt and covert moderating factors that affect the weighted effect (the moderator actually creates subgroups with different effect sizes);

• a small number of studies included in meta-analysis. When a researcher encounters excessive

heterogeneity, such as in a situation where a number of studies have opposite effect direction, or where effect sizes are substantially different, it is necessary to understand whether there is an erroneous inclusion of studies in the meta-analysis. An attempt to combine studies in which completely different parameters were evaluated will lead to huge heterogeneity and will not answer the meta-analysis question. The results of such a meta-analysis will be highly doubtful.

If we believe that there is no error at this stage, it is necessary to look further for the cause of high heterogeneity. For example, when non-standardized parameters are used, it is necessary to try to use standardized instead. If several parameters are measured on different scales or differ significantly on inclusion between studies, standardization makes it possible to smooth out such differences.

The next reason for the high heterogeneity is the presence of covert and overt moderators or confounders. For example, when assessing the prevalence of cardiovascular diseases, sex and age group of patients are obvious moderators. The inclusion of a moderator and the assessment of its impact on the effect and heterogeneity is carried out using meta-regression analysis or meta-regression. Further analysis of subgroups can significantly reduce heterogeneity.

Finally, a small number of studies in a meta-analysis can result in high heterogeneity.

There are 2 main types of heterogeneity by Rucker [10]:

• Heterogeneity due to the design or basic characteristics. The reason is an attempt to combine in a meta-analysis studies of different design (including the study type, the exposure status, the way to results evaluating, the duration of their evaluation and other parameters), as well as studies that are highly heterogeneous by cohorts of patients. This type of heterogeneity can lead to statistical heterogeneity;

• Statistical heterogeneity results from the accuracy of the estimation and variability of the effect size. This type of heterogeneity can already be quantified. One of the reasons (but not in all cases) for statistical heterogeneity can be design-driven heterogeneity.

Methods to measure heterogeneity

Cochran's Q

We looked at two types of models and realized that there is an observable effect 0k of a certain study k, as well as a weighted effect that we calculate 0 for all meta-analysis studies. We also remember that each study has its own weight wk. The deviation of the observed effect from the weighted one can be directed in both the direction with a plus sign or a minus sign. If we square the deviation, it will not depend on the direction.

The sum of the weighted squares of such deviations is called Cochran's Q:

Q = w^ - 0)2 + w2(02 - 0)2 + w3(03 - 0)2 + ... +

+ w (0 - 0)2.

n v n y

We can calculate the deviation of the observed effect from the weighted one for all studies.

The Cochran's Q is distributed as a x2 statistic with K - 1 degrees of freedom, where K is the number of studies in the meta-analysis.

Cochran's Q will grow with an increase in the number of studies in the meta-analysis, as well as with the presence of large studies with many patients in it.

Higgin s & Thompson's I2

I2-statistic

I2 is calculated from Cochran's Q and describes the percentage of heterogeneity which is not caused by sampling error £k. A null hypothesis occurs when there is no heterogeneity, and Cochran's Q follows a distribution of x2 with K - 1 degrees of freedom (expected heterogeneity). But we also have the observed heterogeneity of Q. Then the deviation of the observed heterogeneity from the expected one is:

= Q - (K - 1) Q

and is expressed in unit fractions or a percentage.

Heterogeneity can be qualified as low, moderate, and high, with upper limits of 25%, 50% and 75%, respectively [11].

H2-statistic

H2-statistic calculates the ratio of Q-statistics to K -1. If there is no heterogeneity, then the value tends to 1; higher values indicate the presence of heterogeneity between studies.

H K - 1 •

Heterogeneity of variance t2

The true weighted effect has its own variance t2 and standard deviation t. This parameter is used to evaluate the measure of heterogeneity and has the same dimension as the effect in studies in meta-analysis. If we know the calculated weighted effect size 0 in the meta-analysis, we can estimate the 95% confidence interval (CI) of the true effect as 0 ± 1,96 x t.

SENSITIVITY ANALYSIS

Sensitivity analysis shows how individual studies can influence the weighted effect, and how stable the results of the meta-analysis are. The leave-one-out is one of the most used methods for the evaluation of sensitivity. Each study is excluded from the meta-analysis on an individual basis, then the weighted effect size and heterogeneity are recalculated. Serious changes in the effect size and a decrease in heterogeneity indicate that the excluded study has a significant impact on the

overall result. If at the first stage in the "traffic light" plot the study has a high/some concern risk of bias, then at the stage of using the leave-one-out method it may be an outlier, and it will be necessary to consider the feasibility of its presence in the meta-analysis.

Graphically, the results of the sensitivity analysis are presented in Figure 6.

In the example from Figure 6, the weighted effect obtained in the meta-analysis is the mean difference and amounted to 0.16 [0.1; 0.23]. We see that when studies are excluded from meta-analysis on an individual basis, the weighted effect does not change significantly. At the same time, according to I2 statistics, the exclusion of the Protocol 162A study significantly reduces heterogeneity to 5%. This study requires close attention since its presence causes high heterogeneity in the whole meta-analysis.

Sensitivity analysis evaluates how the weighted effect changes when excluding studies that received "high" and "very high" risk levels when assessing potential risks of bias. If, when one excludes one of these studies, they significantly influence the effect (for example, when, after excluding high-risk studies, 95% CI of the new weighted effect ceases to include the point estimate of the weighted effect before excluding studies), it is necessary to reconsider the need to include high-risk studies in the meta-analysis.

FOREST PLOT

Forest plot is the most common way to summarize the results of a meta-analysis in a single image. It shows the studies included in the analysis, the effect of each one of them, the weighted effect, as well as a set of additional parameters, for example, the weights of each study, parameters of heterogeneity, the type

of chosen statistical model (fixed or random effects). By using forest plots, you can separately duplicate the effect sizes and their 95% CI, as well as the weighted effect and their 95% CI.

The size of the points on the plot that characterize a particular study is often associated with the weight of the study (the points of the largest size are associated with studies with the highest weight, respectively). A common forest plot is shown in Figure 7. The research data are taken from the materials accompanying the meta library of the R programming language [12].

From Figure 7, we can see the names of the studies and the year of results publication, the characteristics of the test and control groups (quantities, means and standard deviations required to calculate the standard error), the effect size (in this case, mean difference, MD) and its 95% CI, weights in fixed and random effects models, weighted effect for both types of models, as well as heterogeneity parameters.

We also see a chart showing all the same effect sizes in studies (represented by squares, the size of which proportional to weights) and the weighted effects for fixed and random effects models (represented by diamond).

A solid vertical line is known as the "line of null effect". If 95% of the CI of individual studies or weighted effects pass through the "line of null effect", the study data is said to be statistically insignificant, and there is a high probability that the observed point estimates are random.

In this example, Floral 1971 was the only study where a significant effect was observed; its weight was the highest in both fixed and random effects models (38.6% and 33.3%, respectively). The fixed effects model showed a significant effect, the means difference

Study

Mean Difference

MD

95%-CI P-value Tau2 Tau 12

Omitting Protocol 154 Omitting Protocol 156 Omitting Protocol 157 Omitting Protocol 162A Omitting Protocol 163 Omitting Protocol 166 Omitting Protocol 303A Omitting Protocol 306

Common effect model

-0.2 -0.1

0

0.14 [0.07; 0.21]

0.15 [0.08; 0.22]

0.16 [0.10; 0.23]

0.18 [0.12; 0.25]

0.16 [0.09; 0.23]

0.17 [0.10; 0.24]

0.15 [0.09; 0.22]

0.17 [0.10; 0.24]

< 0.01 0.0080

< 0.01 0.0076

< 0.01 0.0090

< 0.01 0.0004

< 0.01 0.0099

< 0.01 < 0.01 < 0.01

0.0082 0.0028 0.0099

0.0896 0.0872 0.0949 0.0203 0.0993 0.0904 0.0527 0.0994

45% 47% 51% 5% 51% 49% 27% 51%

0.16 [0.10; 0.23] < 0.01 0.0066 0.0812 43%

0.1 0.2

FIG. 6. Forest plot showing sensitivity analysis, adapted from S. Balduzzi et al. [12]

РИС. 6. Форест-диаграмма, демонстрирующая анализ чувствительности, адаптирована из S. Balduzzi и соавт. [12] Note: MD - Mean difference; Cl - confidence interval; I2 - Higgin's & Thompson's I2 statistic / I2 statistic.

Примечание: study - исследование; MD - Mean difference, разница средних; Cl - confidence interval, доверительный интервал; P-value - значение p; I2 - Higgin's & Thompson's I2 statistic / I2 statistic, ^-статистика Хиггинса и Томпсона / ^-статистика; common (fixed) effect model - модель с фиксированными эффектами.

Study

Davis 1973 Florell 1971 Gruen 1975 Hart 1975 Wilson 1977

Common effect model Random effects model

Heterogeneity: I2 = 29%, t2 = 0.2742, p = 0.23

Experimental Control

Total Mean SD Total Mean SD

13 5.00 4.7000 13 6.50 3.8000

30 4.90 1.7100 50 6.10 2.3000

35 22.50 3.4400 35 24.90 10.6500

20 12.50 1.4700 20 12.30 1.6600

8 6.50 0.7600 8 7.38 1.4100

106

126

Mean Difference

О

Weight Weight

MD 95%-CI (common) (random)

-1.50 [-4.79; 1.79] 2.8% 5.2%

-1.20 [-2.08; -0.32] 38.6% 33.3%

-2.40 [-6.11; 1.31] 2.2% 4.1%

0.20 [-0.77; 1.17] 31.9% 30.6%

-0.88 [-1.99; 0.23] 24.5% 26.8%

-0.71 [-1.26;-0.16] 100.0% _

-0.75 [-1.53; 0.03] - 100.0%

FIG. 7. Forest plot showing the weighted effect, adapted from S. Balduzzi et al. [12]

РИС. 7. Forest-диаграмма, демонстрирующая взвешенный эффект, адаптирована из S. Balduzzi и соавт. [12]

Note: SD - standard deviation; MD - mean difference; CI - confidence interval; I2 - Higgin's & Thompson's I2 statistic / I2 statistic; т2 - Tau-squared. Примечание: study - исследование; experimental - экспериментальная группа; control - контрольная группа; total - общее значение; mean -среднее значение; SD - standard deviation, стандартное отклонение; MD - mean difference, разница средних; CI - confidence interval, доверительный интервал; weight (common) - веса в модели с фиксированными эффектами; weight (random) - веса в модели со случайными эффектами; common (fixed) effect model - модель с фиксированными эффектами; random effect model - модель со случайными эффектами; heterogeneity - гетерогенность; I2 - Higgin's & Thompson's I2 statistic / I2 statistic, ^-статистика Хиггинса и Томпсона / ^-статистика; т2 - Tau-squared, Тау-квадрат; P-value - значение p.

was -0.71 [-1.26; -0.16], while the random effects model was insignificant, since the point estimate was -0.75, and 95% CI included zero [-1.53; 0.03] according to the results of the meta-analysis

CONCLUSION

In this section of the article, we have introduced the reader to the stages of including studies in meta-analysis, reviewed the existing guidelines that you need to familiarize with when writing a meta-analysis, analyzed in detail the process of creating weights, various types of models used in meta-analyses. We also

AUTHOR CONTRIBUTIONS

Alexander Yu. Suvorov, Irina V. Latushkina and Kseniya A. Gulyaeva equally contributed to this work and should be considered the first co-authors. Alexander Yu. Suvorov, Irina V. Latushkina, Kseniya A. Gulyaeva, Nikolay M. Bulanov, Maria Yu. Nadinskaia and Alexey A. Zaikin participated in writing the text of the manuscript. Alexander Yu. Suvorov, Irina V. Latushkina, Nikolay M. Bulanov and Alexey A. Zaikin searched and analyzed the literature on the topic of the review. Alexander Yu. Suvorov developed the general concept of the article, Alexey A. Zaikin supervised writing the text of the manuscript. All authors participated in the discussion and editing of the work. All authors approved the final version of the publication.

became acquainted with the definition of heterogeneity and understood how it is calculated as well as the main plot published in meta-analyses such as the forest plot and the leave-one-out plot for sensitivity analysis were presented.

In the next article, we will consider how to analyze subgroups, familiarize with meta-regression, and learn how to evaluate the publication bias, visually and mathematically. Additionally, we will recall the most common ways of evaluating the meta-analysis effect and focus on standardized indicators and the evaluation of the standard error for them.

ВКЛАД АВТОРОВ

А.Ю. Суворов, И.В. Латушкина и К.А. Гуляева в равной степени внесли вклад в эту работу и должны считаться первыми соавторами. А.Ю. Суворов, И.В. Латушкина, К.А. Гуляева, Н.М. Буланов, М.Ю. Надинская и А.А. Заикин участвовали в написании текста рукописи. А.Ю. Суворов, И.В. Латушкина, Н.М. Буланов и А.А. Заикин выполняли поиск и анализ литературы по теме обзора. А.Ю. Суворов разработал общую концепцию статьи, А.А. Заикин осуществлял руководство ее написанием. Все авторы участвовали в обсуждении и редактировании работы. Все авторы утвердили окончательную версию публикации.

REFERENCES / ЛИТЕРАТУРА

1 Sackett D.L., Rosenberg W.M., Gray J.A., et al. Evidence based medicine: what it is and what it isn't. BMJ. 1996 Jan 13; 312(7023): 71-72. https://doi.org/10.1136/bmj.312.7023.71. PMID: 8555924

2 Page M.J., McKenzie J.E., Bossuyt P.M., et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews BMJ 2021; 372: n71. https://doi.org/10.1136/bmj.n71

3 Sterne J.A.C., Savovic J., Page M.J., et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019

Aug 28; 366: 14898. https://doi.org/10.1136/bmj.l4898. PMID: 31462531

4 Sterne J.A., Hernán M.A., Reeves B.C., et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016 Oct 12; 355: i4919. https://doi.org/10.1136/bmj.i4919. PMID: 27733354

5 McGuinness L.A., Higgins J.P.T. Risk-of-bias VISualization (robvis): An R package and Shiny web app for visualizing risk-of-bias assessments. Res Synth Methods. 2021 Jan; 12(1):

55-61. https://doi.org/10.1002/jrsm.1411. Epub 2020 May 6. PMID: 32336025

6 Suvorov A.Yu., Bulanov N.M., Shvedova A.N., et al. Statistical hypothesis testing: general approach in medical research. Sechenov Medical Journal. 2022; 13(1): 4-13. https://doi. org/10.47093/2218-7332.2022.426.08

7 Viechtbauer W. Bias and efficiency of meta-analytic variance estimators in the random-effects model. Journal of Educational and Behavioral Statistics, 2005; 30(3): 261-293. https://doi. org/10.3102/10769986030003261

8 Veroniki A.A, Jackson D., Viechtbauer W., et al. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Res Synth Methods. 2016 Mar; 7(1): 55-79. https://doi.org/10.1002/jrsm.1164. Epub 2015 Sep 2. PMID: 26332144

9 LanganD., Higgins J.P.T., JacksonD., et al. A comparison of heterogeneity variance estimators in simulated random-effects metaanalyses. Res Synth Methods. 2019 Mar; 10(1): 83-98. https:// doi.org/10.1002/jrsm.1316. Epub 2018 Sep 6. PMID: 30067315

10 Rucker G., Schwarzer G., Carpenter J.R., Schumacher M. Undue reliance on I(2) in assessing heterogeneity may mislead. BMC Med Res Methodol. 2008 Nov 27; 8: 79. https://doi.org/10.1186/1471-2288-8-79. PMID: 19036172

11 Higgins J.P., Thompson S.G. Quantifying heterogeneity in a metaanalysis. Stat Med. 2002 Jun 15; 21(11): 1539-1558. https://doi. org/10.1002/sim.1186. PMID: 12111919

12 Balduzzi S., Rucker G., Schwarzer G. How to perform a metaanalysis with R: a practical tutorial. Evid Based Ment Health. 2019 Nov; 22(4): 153-160. https://doi.org/10.1136/ebmen-tal-2019-300117. Epub 2019 Sep 28. PMID: 31563865

INFORMATION ABOUT THE AUTHORS / ИНФОРМАЦИЯ ОБ АВТОРАХ

Alexander Yu. Suvorov, Cand. of Sci. (Medicine), Chief Statistician, Centre for Analysis of Complex Systems, Sechenov First Moscow State Medical University (Sechenov University). ORCID: https://orcid.org/0000-0002-2224-0019

Irina V. LatushkinaH, junior researcher, Centre for Analysis of Complex Systems, Sechenov First Moscow State Medical University (Sechenov University) ORCID: https://orcid.org/0000-0001-8885-6062

Kseniya A. Gulyaeva, postgraduate student, Department of Propaedeutics of Internal Diseases, Gastroenterology and Hepatology, Sechenov First Moscow State Medical University (Sechenov University)

ORCID: https://orcid.org/0000-0002-3462-0123

Nikolay M. Bulanov, Cand. of Sci. (Medicine), Associate Professor, Department of Internal, Occupational Diseases and Rheumatology, Sechenov First Moscow State Medical University (Sechenov University).

ORCID: https://orcid.org/0000-0002-3989-2590

Maria Yu. Nadinskaia, Cand. of Sci. (Medicine), Associate Professor, Department of Propaedeutics of Internal Diseases, Gastroenterology and Hepatology, Sechenov First Moscow State Medical University (Sechenov University). ORCID: https://orcid.org/0000-0002-1210-2528

Alexey A. Zaikin, Cand. of Sci. (Physics and Mathematics), Deputy Director, Centre for Analysis of Complex Systems, Sechenov First Moscow State Medical University (Sechenov University).

ORCID: https://orcid.org/0000-0001-7540-1130

Суворов Александр Юрьевич, канд. мед. наук, главный статистик Центра анализа сложных систем ФГАОУ ВО «Первый МГМУ им. И.М. Сеченова» Минздрава России (Сеченовский Университет).

ОКСЮ: https://orcid.org/0000-0002-2224-0019

Латушкина Ирина Викторовнан, младший научный сотрудник Центра анализа сложных систем ФГАОУ ВО «Первый МГМУ им. И.М. Сеченова» Минздрава России (Сеченовский Университет).

ОКСЮ: https://orcid.org/0000-0001-8885-6062

Гуляева Ксения Александровна, аспирант кафедры пропедевтики внутренних болезней, гастроэнтерологии и гепатоло-гии ФГАОУ ВО «Первый МГМУ им. И.М. Сеченова» Минздрава России (Сеченовский Университет). ОКСЮ: https://orcid.org/0000-0002-3462-0123

Буланов Николай Михайлович, канд. мед. наук, доцент кафедры внутренних, профессиональных болезней и ревматологии ФГАОУ ВО «Первый МГМУ им. И.М. Сеченова» Минздрава России (Сеченовский Университет). ОКСЮ: https://orcid.org/0000-0002-3989-2590

Надинская Мария Юрьевна, канд. мед. наук, доцент кафедры пропедевтики внутренних болезней, гастроэнтерологии и гепатологии ФГАОУ ВО «Первый МГМУ им. И.М. Сеченова» Минздрава России (Сеченовский Университет). ОКСЮ: https://orcid.org/0000-0002-1210-2528

Заикин Алексей Анатольевич, канд. физ-мат. наук, заместитель директора Центра анализа сложных систем ФГАОУ ВО «Первый МГМУ им. И.М. Сеченова» Минздрава России (Сеченовский Университет). ОКСЮ: https://orcid.org/0000-0001-7540-1130

н Corresponding author / Автор, ответственный за переписку