CC BY
20DOI: https://doi.org/10.21323/2414-438X-2021-6-2-151-162
/©¡creative ^commons
Available online at https://www.meatjournal.ru/jour Original scientific article
METHODS FOR NONPARAMETRIC STATISTICS R ed 2404 2021
IN SCIENTIFIC RESEARCH. OVERVIEW. PART 1 ^PfpUbZ* lO.SOfi
Marina A. Nikitina*, Irina M. Chernukha
V. M. Gorbatov Federal Research Center for Food Systems of Russian Academy of Sciences, Moscow, Russia
Keywords: nonparametric statistics, Rosenbaum Q-test, Mann-Whitney U-test, sign G-test, Wilcoxon T-test Abstract
Daily, researcher faces the need to compare two or more observation groups obtained under different conditions in order to confirm or argue against a scientific hypothesis. At this stage, it is necessary to choose the right method for statistical analysis. If the statistical prerequisites are not met, it is advisable to choose nonparametric analysis. Statistical analysis consists of two stages: estimating model parameters and testing statistical hypotheses. After that, the interpretation of the mathematical processing results in the context of the research object is mandatory. The article provides an overview of two groups of nonparametric tests: 1) to identify differences in indicator distribution; 2) to assess shift reliability in the values of the studied indicator. The first group includes: 1) Rosenbaum Q-test, which is used to assess the differences by the level of any quantified indicator between two unrelated samplings; 2) Mann-Whitney U-test, which is required to test the statistical homogeneity hypothesis of two unrelated samplings, i. e. to assess the differences by the level of any quantified indicator between two samplings. The second group includes sign G-test and Wilcoxon T-test intended to determine the shift reliability of the related samplings, for example, when measuring the indicator in the same group of subjects before and after some exposure. Examples are given; step-by-step application of each test is described. The first part of the article describes simple nonparametric methods. The second part describes nonparametric tests for testing hypotheses of distribution type (Pearson's chi-squared test, Kolmogorov test) and nonparametric tests for testing hypotheses of sampling homogeneity (Pearson's chi-squared test for testing sampling homogeneity, Kolmogorov-Smirnov test).
Funding:
The research was supported by state assignment of V. M. Gorbatov Federal Research Centre for Food Systems of RAS, scientific research No. FNEN-2019-0008.
Introduction
The starting point for state-of-the-art statistical methods development is 1900. In 1901, the English mathematician, statistician, biologist and philosopher Karl Pearson, together with Francis Galton and Walter Weldon, founded the Biometrika journal (https://academic.oup. com/biomet) to promote the introduction of mathematical methods in biology. In 1925, Karl Pearson created another journal, Annals of Human Genetics (https://on-linelibrary.wiley.com/journal/14691809) dedicated to human genetics.
First third of the twentieth century passed under the sign of parametric statistics, which is defined as a section of statistics "assuming that the sampling belongs to the general population, which may be sufficiently accurately and adequately modeled by a probability distribution with specified set of parameters" [1]. Its main object is sampling from distributions described by one or several parameters. Methods based on the data analysis from parametric distributions described by the Pearson curves were studied. The normal distribution (Gaussian distribution) was the most popular one. Exponential and logarithmic normal distributions, Weibull-Gnedenko distributions, gamma distributions, binomial and hypergeometric distributions, Poisson distributions, etc. were used. Pearson test, Student test, and Fisher test were used to test hypotheses. The method of maximum likelihood and analysis of variance
were proposed, and the main ideas for experiment design were stated [1-7].
Simultaneously with parametric statistics, in the works of Charles Edward Spearman and Maurice George Kendall, the first nonparametric methods based on rank correlation coefficients were introduced. Now the above methods are named after these statisticians. But nonparametric statistics became a noticeable part of statistics only since the second third of the twentieth century. Originally, nonpara-metric methods were designed to test statistical hypotheses (about one-dimensional probability distributions). The most famous nonparametric tests are Kolmogorov-Smirnov tests developed in 1930s, Wilcoxon rank tests and the Mann-Whitney U-test developed in 1940s-1950s, and, of course, rank correlation coefficients (1904-1930) by C. Spearman and M. Kendall [8-24].
Nonparametric tests are tests that do not include distribution parameters in the calculation formula and do not suppose knowledge of the distribution function. These tests use frequencies or ranks. They are used when comparing samplings with nominal and serial indicators, as well as for samplings measured on quantitative scales for which the distribution law is unknown or differs from normal one.
A feature of the nonparametric methods is that the probability distribution is considered completely unknown, and the problems are stated only in terms of dif-
FOR CITATION: Nikitina, M.A., Chernukna, I.M. (2021). Methods for nonparametric statistics in scientific research. Overview. Part 1. Theory and
practice of meat processing, 6(2), 151-162. https://doi.org/10.21323/2414-438X-2021-6-2-151-162
ferences between classes or within classes of unknown distributions.
If an alternative hypothesis contains a statement about the expected direction of differences, then such a hypothesis is called "directional", otherwise the hypothesis is called "non-directional".
The level of statistical significance or critical values of tests are determined differently when testing directional and non-directional hypotheses.
When testing the directional hypothesis, one-sided test is used, while two-sided test is used for the non-directional hypothesis. The two-sided test is more stringent because it tests for differences in both directions, and therefore the empirical value of test that previously met a significance level of < 0.05 now corresponds to a level of < 0.1.
Many problems in research and practical work are associated with the need to compare two or more groups of observations obtained under different conditions, as well as the need to classify observations (rank them to one of several classes). The solution for such problems is possible only when it is determined what is the difference (or identity) for the groups of observations. When the compared sets of numbers are so different that they do not overlap, the researcher has no doubt that the sets are different. Difficulties begin when the sets overlap.
In the first part of the article, we will discuss two groups of nonparametric tests: 1) to identify differences in indicator distribution; 2) to estimate shift reliability in the values of the studied indicator.
The aim of the research was to determine the application limits of the tests and methods in nonparametric statistics.
Nonparametric tests to identify differences
in indicator disteibution
This group of testr intludes Rosenbaum Q-test [21], Mann-Whitney U-test [20],
1. Rosenbaum Q-test
The test it used to assess the differences by the level of any quantified indicator between two unrelated samplings [21].
a) b)
Figure 1 shows a possible arrangement of two sets (samplings). Two compared sets are ideally arranged if they may be represented as in Figure 1 (a): Nl is the number of observations in one sampling exceeding the maximum value in the other sampling; N2 is the number of observations of one sampling, the values of which are less than the minimum value in the other sampling.
In case (a), there is no overlap between the values of both sets. Therefore, there is a statistically significant difference between the two sets (i. e., the difference is significant). In Figure 1b, both sets are at the same level. Therefore, the difference is not significant. If there are equal values in the compared sets, they should be placed exactly opposite each other. In Figure 1c, sets partially overlap, but the first set is higher than the second one.
Let's explain why N=0 and N2=0 in Figure 1b. Nj are numbers from sampling 1 that are higher than the maximum value in sampling 2. In Figure 1b, there are no such numbers in sampling 2. N2 are numbers from sampling 2 that are less than the minimum value in sampling 1. In Figure 1b, there are no such numbers in sampling 1. Thus, Nj and N2 in Figure 1b are equal to 0.
The Q-value (Rosenbaum Q-test) is equal to the sum of Nj and N2. The higher it is, the differences are more reliable, i. e. E2=jN (Figure 1a) > £2= N (Figure 1c) > £2= N (Figure 1b). Experimental value of Rosenbaum Q-test is calculated by the formula:
Q p = N. + N, (1)
^ exp 12 x '
The following hypotheses are tested: H0: Indicator level in sampling 1 does not exceed indicator level in sampling 2.
H: Indicator levelin sampling 1 exceeds indieator level in sampling 2.
Applrearion of Rosenbaum Q-test
1. Arrange the valur s oft iedicato- in both samplinsr in dercending tor asctnding) orfer. Consiter stmplmg 1 the one that is supposed tt include higher values, and sampling 2 the one that is supposed to include lower values (Figure 1c).
Figure 1. Possible ratios for sets of values in two samplings. Nt is the area of values in the 1st set, which are higher than the maximum value in the 2n0 set; N2 is thr area of values in tire 2nd set,which are less thrn the minimum value in the lit set; oveolepping areas
oS two sets are marked with hatching
2. Determine the highest (maximum) value in sampling 2.
3. Count the number of values in sampling 1 that are higher than the maximum value in sampling 2 (N)
4. Determine the lowest (minimum) value in sampling 1.
5. Count the number of values in sampling 2 that are lower than the minimum value in sampling 1 (N2).
6. Calculate Qexp by the formula (1).
7. In the table of critical values presented in [7, 21, 25, 26] for given ni and n2 and the significance level a, find the critical value Q .
cr
8. If Qexp is equal to Qcr or exceeds it, H0 is rejected.
Limitations of Rosenbaum Q-test
1. Measurement may be carried out on a scale of order, intervals and ratios.
2. Samplings must be unrelated.
3. Each sampling must contain at least 11 observations. Furthermore, the volumes of samplings should be approximately the same:
a) if both samplings contain less than 50 observations, then the difference between the volumes of samplings should be not more than 10 observations;
b) if each sampling contains more than 50 observations, but less than 100 observations, then the difference between the volumes of samplings should be not more than 20 observations;
c) if each sampling contains more than 100 observations, then it is allowed that one of the samplings is 1.5 to 2 times larger than the other one.
4. If the numbers of observations nl and n2 are > 26, the following values may be used:
Q f 8 at a < 0.05 Qcr 110 at a < 0.01
5. The ranges of value distribution for the two samplings should not overlap. In this case, the Q-test is not applicable. Therefore, Mann-Whitney U-test should be used.
Example
Given data: Heart rate was measured in two groups of animals. One group (11 animals) includes animals before the experiment, the other group (12 animals) includes animals after the experiment. The results are presented in Table 1. Is it possible to state that one of the groups is superior to the other one in terms of heart rate at the significance level? [215].
Solution. All the Q-test limitations are met, so Ro sen-baum Q-test may be used to identify differences in indicator distribution.
Let's arrange the values in both samplings in ascending order, and then state the hypotheses:
H0: The first group of animals does not exceed the second group of animals in heart rate.
H The first group of animals exceeds the second group of animals in heart rate.
The calculations are presented in Table 2. The first set is the one that is "higher" — set No. Nl, and the second set is the one that is "lower" — set No. N. The maximum value
in the second sampling is 122 and the minimum value in
the first sampling is 92.
Table 1. Given data for the problem
1st group of animals 2nd group of animals
No. 1 Heart rate 92 ÛQ No. 1 Heart rate 77 Q 1
2 3 A 98 104 2 3 A 81 81 Q A
4 5 t: 107 107 1 T) 4 5 ti 84 89 QC
6 7 Q 122 122 1 T) 6 7 Q 95 101 1 ni
0 9 1 n 122 127 1 xx 8 9 1 n 103 107
1U ii 133 137 10 11 12 110 114 122
From Table 2, we determine the number of values in the first set (137, 133, 127) that are higher than the maximum value in the second set (Nj = 3) and the number of values in the second set (899, 84, 81, 81, 77) that are lower than the minimum value in the firct set (N2 = 5).
We calculate Q accordmg to the formuta (1): Q = N. + N. = e + 5 = 8. From the table of critical val-
9exp 1 2
ues presented in [7, 21, 25, 26], we can find critical values for tine given sampling volume and the signifi cance level. Ae n. = 11, sc, = 12 and we have Q = 6.
1 2 ^cr
Table 2. Calculation results
No. i Heart rate 1A No. Heart rate
1 2 3 13/ T 133 N 127 \
4 C 122 1 T) 1 122
5 6 1222 1222 11/1
n 1 f\H 2 3 A 114 110 1 (\7
7 8 o 107 107 4 IV/
9 104 5 /r 103 11t 1
10 98 6 7 101 QC
11 92 7 95
N 1 8 9 1 n 89 84 Q 1
N2 1 10 11 r 12 81 81 77
Let's plot theaxis of significance:
Q = 6 Q p =8
cr ^exp
Since Q > Qcr, H0 is rejected. And is accepted. Thus, the second group of animals is superior to the animals of the first group in heart rate (at a < 0.05).
In [27], the authors applied the method of mathematical statistics (Rosenbaum Q-test) when finding the optimal technology for developing professional competencies among the target group of students studying business informatics. In [28], the exercise training effect on the health and condition in two groups of women of different ages was assessed by Student t-test (t) and Rosenbaum Q-test (Q). In [29], to compare and evaluate the data in five groups of 110 students, in a study on how English can be effectively implemented in the training of students of technical specialties for developing professional skills, the statistical method with Rosenbaum Q-test was used.
2. Mann-Whitney U-test
The test is designed to test the statistical homogeneity hypothesis of two unrelated samplings, i. e. to assess the differences by the level of any quantified indicator between two samplings [20]. It allows to differentiate between small samplings when nl, n2 > 3 or nl = 2, n2 > 5 (nl and n2 are volumes of samplings), and is more powerful than the Rosenbaum Q-test. The Rosenbaum Q-test is designed for low-volume samplings. Thus, Rosenbaum Q-test is a particular case of Whitney-Mann test.
Two samplings, x1, ..., xn (sampling X) and y,, ..., yn (sampling Y) with volumes of n1 and n2 are studied. Let's denote the distribution law of the first sampling by F, and the distribution law of the second sampling by G. The laws of distributions F and G are continuous. From here it follows with probability 1, that among the numbers x1, ., xn and y,, ., yn, there are no overlapped ones.
Let's test the null hypothesis H0: F = G.
All possibilities of F ^ G may be the alternative hypotheses. However, the test is not able to detect all possible deviations from H0. First of all, this test is intended to test the H0 hypothesis against the alternative of F > G (right-sided alternative, "overflow" of probabilities to the right) or the alternative of F < G (left-sided alternative, "shift" of probabilities to the left). Combining both possibilities (two-sided alternative) may also be studied.
The Mann-Whitney U-test is based on a pairwise comparison of the results from the first and the second set 1 set 1
set 2
sampling. For values of x. < y , an event means "success", and any event with x. > y, means "failure". By changing
1 from 1 to n1 and j from 1 to n2, we have n1 x n2 pairwise comparisons of values in samplings X and Y.
Let's denote the number of successes in these pairwise comparisons by U. U-value can take any integer value from 0 to n1xn2. The random U-value is called the Mann-Whitney statistics.
This test is particularly effective when testing an alternative hypothesis of dominance, and its particular case, i. e. the hypothesis of the right shift, when the values in the first sampling are higher than the values in the second sampling.
The test determines if the overlapping values area between the two sets is small enough. The first set, sampling or group is the set of values in which the values, according to preliminary estimates, are higher. And the 2nd set is the one where the values are supposed to be lower. The smaller the area of overlapping values, the more likely the differences are significant. These differences are sometimes referred to as differences in the location of the two samplings. The experimental value Uexp reflects how large is the overlapping area between sets. Therefore, the smaller Uex, the more likely the differences are significant. Figure
2 shows three of the many possible relationships between two sets of values.
In the first variant (Figure 2a), the second set is lower than the first one, and the sets almost do not overlap. This means that the values are almost different. The overlapping area is too small to hide the differences between sets. The differences between seem to be significant. In the second variant (Figure 2b), the second set is also lower than the first one, but the area of overlapping values between the two sets is quite large. It may not yet reach a critical value, when the differences have to be recognized as insignificant. Finally, in the third variant (Figure 2c), the second set is lower than the first one, but the overlapping area is so large that the differences between the sets are insignificant.
The following hypotheses are tested:
H0: indicator level in group 2 is not lower than indicator level in group 1.
H indicator level in group i is lower than indicator level in group 1.
set 1
set 2
set 2
a) " b) c)
Figure 2. Possible ratios for sets of values in two samplings; overlapping areas are marked with hatching
Application of Mann-Whitney U-test
1. Pool the values for both groups in the order of indicator rising, regardless of what sampling they belong to.
2. Rank the values by assigning a lower rank to a lower value. Total ranks (nl + n2).
3. Calculate the sum of the ranks for the values in the first sampling and the sum of the ranks for the values in the second sampling. Determine the larger of the two sums of the ranks.
4. Determine U value by the formula:
exp '
exp
TT nx ' (nx + T
U = n, • n + ——P--T
exp 12 2 x
(2)
where n, n2 are volumes of samplings 1 and 2; Tx is the larger of the two rank sums; n is the volume of sampling with the larger sum of the ranks.
5. Determine the critical values Ur for the given nj, n2 and the significance level a. If Uxp > Ucr, then H0 is accepted. If U < U , then H. is rejected. The lower the U-value,
exp cr 0 J
the higher the reliability of the differences.
Limitations of Mann-Whitney U-test
1. Each sampling must contain at least 3 observations: n, n2 > 3; it is allowed that one of the sampling includes 2 observations, but then the second one should include at least 5 observations.
2. Each sampling should contain no more than 60 observations: n, n2 < 60.
Example
Given data. In the control group (without treatment) and in the experimental group (with treatment), the death of animals was established in time intervals (minutes) after the introduction of a toxic substance [7]. The results are shown in Table 3.
Table 3. Given data for the problem
Sampling No. 123456789
No. 1 — without treatment 39 37 45 5 27 27 31
No. 2 — with treatment 45 9 67 42 42 31 34 49 99
Let's state the hypotheses:
H0: the difference between the indicators in the control group and the experimental group is statistically insignificant.
H the difference between the indicators in the control group and the experimental group is statistically significant.
Solution. In this case, the Rosenbaum Q-test is not applicable, since the volume of both samplings is less than 11. Let's apply the Mann-Whitney U-test.
Let's pool both samplings arranging their values in ascending order, while indicating which sampling they belong to and ranking them. The data are summarized in Table 4.
Rank is a serial number. The lowest value has rank 1.
The value of 5 — rank 1.
The value of 9 — rank 2.
We have the value of 27 twice, so it takes positions (ranks) 3 and 4. We add (3 and 4) and divide by 2 (since 27 occurs twice in sampling). Thus (3 + 4)/2 = 3.5. The rank of 27 is 3.5.
The value of 31 occurs twice in sampling taking positions 5 and 6. Similarly, we add 5 and 6 and divide by 2, so we get rank 5.5, etc. Table 4. Pooled data from two samplings
Sampling No. Values rank Sampling No. Values rank
1 5 1 1 39 9
2 9 2 2 42 10.5
1 27 3.5 2 42 10.5
1 27 3.5 1 45 12.5
1 31 5.5 2 45 12.5
2 31 5.5 2 49 14
2 34 7 2 67 15
1 37 8 2 99 16
Let's check the correctness of the ranking: £(R.) = 1 + 2 + 3.5 + 3.5 + 5.5 + 5.5 + 7+ ... + 14 + 15 + 16 = 136
Using the formula that calculates the sum of ranks in sampling, we determine the correctness of the ranks. As a result:
S^Nif+iL !6<1iiU=136
Thus, the ranks are assigned correctly.
Calculating the ranks in the first and the second sampling:
Rj = 1 + 3.5 + 3.5 + 5.5 + 8 + 9 + 12.5 = 43
R2 = 2 + 5.5 + 7 + 10.5 + 10.5 + 12.5 + 14 + 15 + 16 = 93
Checking: 43 + 93 = 136.
The larger of the two rank sums Tx = 93 (corresponds to the second sampling n2 = 9), i. e. nx = 9.
Let's find the experimental value of test by the formula:
e n x-(nx + 1) co 9 • (9 + 1) Q, 1C U = n, • n, + ——x--T= 7 • 9 +----93 = 15
exp 1 2 x 2
We can find the critical -value of test according to the table presented in [7,21,25, 2(5]. In this ease, we ¡ere testing the non-directional alternative hypothesis. With nl = 7 and
n, = 9, U = 12 foil a < 0.05.
2 cr
Let's plot the axis of significance:
U = 12 U = 15
cr exp
Sinee Uexp e Up, H0 is accepted. Thus, the significant; differences between the indicators of fhe control and the ex-per imental group leas not been established.
In [30], the protein damage of individual protease inhibitors of the serpin superfamily in CRSwNP was studied. The damage of protease inhibitors by fibrinolysis in tissues was compared with exosome samples to assess the potential of "liquid biopsy" non-invasive exosomes for CRSwNP. Semi-quantitative analysis of Western blots was performed using the Mann-Whitney U-test. Effectiveness comparison of the standard simulation training in emergency care with a more comprehensive simulation program for emergency care is shown by the authors in [31]. The one-sided t-test and the Mann-Whitney U-test were used to analyze the Objective Structured Clinical Examination (OSCE) scores. The assessment of the effect of ketorolac intravenous injections on pain relief before and after using a tourniquet in orthopedic surgery on the femur and tibia was shown in [32]. This double-blind clinical study enrolled one hundred patients aged 15 to 75, candidates for hip and lower leg; surgery. Visual pain tests during recovery were measured and analyzed using descriptive (mean and percentage ) and logical statistics (Mann-Whitney U-test). The study [33] assessed the implementation of patient-centered interventions. 34 patients were examined with elective total knee arthroplasty. Endpoints were duration of induction period (primary), duration of hospital stay, pain at rest and pain at physical activity on postoperative day 1 (numeric analogue scale), and surgical release progress (MBF) on postoperative days 1, 3, and 6 (secondary). Group inferiority comparisons were made using the Wilcoxon T-test an d Mann-Whitney U-test. Work [34] studied MRI of sodium (23Na) at a field strength of 7 Tesla (T) as a biomarker of tumor stage, isocitrate dehydrogenase (IDH) mutatiyn, and O6-methylguanine-DNA methyltransferase (M GMTf promoter methylation in patients with glioma. The diffefi ence in median 23Na concentration throughout the tumor area was compared between gliomas with IDH mutation and wild-type IDH, as well as between glioblastomas with methylated and non-methylated MGMT using the MannWhitney U-test.
Nonparametric tests for assessing shift reliability in the values of the studied indicator This group of tests includes sign G-test and Wilcoxon T-test.
Let a random variable have a distribution F(x), and another random variable has a distribution S(x), and S(x)=F(x - A), where A is an abstract parameter characterizing the state of the object under study.
Two samplings, x1, ..., xn (sampling from F(x)) and y,, ..., yn (sampling from S(x)), with volumes n are studied. Furthermore, F(x) and S(x) are unknown distribution functions.
It is required to test the null hypothesis H0: F(x) = S(x), i. e. A = 0. Three types of alternative hypotheses may be stated: Directional hypotheses:
a) H A > 0;
b) Hj2: A < 0;
non-directional hypothesis: c) H3: A * 0.
The following hypotheses are tested:
H0: indicator level in group 2 is not lower than indicator level in group 1.
H indicator level in group 2 is lower than indicator level in group 1.
Thus, it is known that if the impact changes the indicator distribution, then the impact shifts it in a quite definite way, i. e. by the value of A. If the shifts are compared in two unrelated samplings, for example, in the experimental and the control groups of subjects, then the Rosenbaum Q-test and the Mann-Whitney U-test should be used. To determine the shift reliability of related samplings, for example, when measuring indicator in the same group of subjects before and after some exposure, the sign G-test and the Wilcoxon T-test should be used.
1. Sign G-test
Sign G-test is intended to establish a general shift direction of the studied indicator. It allows to determine, in which direction the values of the indicator in sampling gener ally change dur ing thf transition from the first measuring to the iecond one, i. e. whether the indicators change in thy direction of improvement/increasn or, ton-versely, in the direction of deteriotationi'decrease. Shift is ihe difference in the values of indicators measured after a certaan impact and before it [35, 36].
The sygn G-test fs applicable both to those shifts that can only be determined qualitatively (for example, a change from a negative attitude too n positive one) and to those shifts that ran be quantified.
yh[fts that seem no be predominant are usual, and shifts in a rarer direction are unusual. If [he indicators do not increase or decrease, then such ihiftn aire called zero shift) and they are excluded erom the sign G(tese. Furthermore, thin number oi compated pairs is reduced by the number of tuch zero shifts.
The sign G-test determines whether there are too many unusual shrifts tbserved to con sidnr shift in usual dire cti on to be predominant. The less unusual shifts, the more likely the prevalence of usual shift is reliable. Figure 3 shows usual shifts as a light area, and unusual shifts as a dark area.
+ + +
Figure 3. A graphic representation of positive and negative shifts in the form of different areas: light area is positive shifts; dark area is negative shifts [26]
The following hypotheses are tested: Hf predominance of usual shift direction is random; HG:p red ominance of usual shift direction is not random.
Limitations of sign G-test
The number of observations in both measurements should be not less than 5 and not more than 300.
Application of sign G-test
1. Check if the limitations are met.
2. Enter the measurement data in the table (Table 5): Table 5. The filled table
No. Pre-exposure values Post-exposure values «After to before» shift
1
2
n
3. Shift is not quantified, but the difference sign "+" or "-" is added. If the difference is zero, then zero is written in the table.
4. Calculate the number of zero responses n0 and exclude them from consideration. As a result, the volume of sampling will decrease by the number of zero responses: n = n - n0.
5. Determine the predominant direction of change. Shifts in the predominant direction are "usual".
6. Determine the number of unusual shifts. Consider this number as the experimental value of G-test (G).
7. Determine the critical value Gr for a given n and a specified significance level a.
8. Compare G and G . If G < G , H. is rejected, i. e.
r exp cr exp cr 0 J
shift in the typical direction may be considered reliable. Example
Given data: A group of 10 students completed a mathematics test. After checking the examination work, this group was asked to study an electronic manual on the same topic, after which a repeated test was carried out, which was assessed on a five-point scale. Has the knowledge of the students improved after studying the manual? [26].
Solution. The results of double examination work; represent measurements in a serial scale. The volume of sampling is higher than 5, so the use of the sign G-test is possible. The results are summarized in Table 6:
Table 6. Given data for the problem
No. First testing Second testing Sign of the difference
1 3 4 +
2 2 3 +
3 2 2 0
4 5 4 3 3 2 —
6 3 4 +
7 4 4 +
8 2 3 +
9 2 2 0
10 3 4 +
The number of zero responses n0 = 2. Let's exclude them from consideration, therefore, volume of sampling n = 10-2 = 8.
The number of positive shifts is 6, and the number of negative shifts is 2. This means that the predominant direction is the positive one, therefore, shifts in this direction are usual, and shifts in the negative direction are unusual.
Let's state the hypotheses:
H0: predominance of usual shift direction is random, i. e. students' knowledge did not improve after studying; the manual.
H,: predominance of usual shift direction is not random, i. e. students' knowledge improved after studying the manual (thift in the positive direction is reliable).
Let's tost at the significance level a < 0.05.
The number 01e unusual shifts is 2i i. e. (according to item 6).
For n = 8 and a < 0.05 we can find from the table of critical, values [7, 21,25, 26].
Let's plot the axis of significance:
Since G > G , there is no reason to reject He, end the
exp cr J 0
predominance of usual shift direction is random, so the students' knowledge did not improve after studying the manual
2. Wtlcoxon T-test
The test is used tf compare values measured under two different condition s in tine same sampling. ft allows to establish not only the direction of changes, but also their intensity. It may be used to determine if the shift of indicators in one direction is more intense than in the other direction. Test; is applicable in cases where indicators are measured tit le asi on a seri al scale, and the shifts between the second and first measurements can also b e ordered. Therefore, they must vary within a considerably wide range [22, 23, 24].
The method compares the shift intensity in one direction with the shift intensity in another direction in absolute value. To do this, you need to rank all absolute shifts, and then summarize the ranks of shifts separately in the positive and negative directions. If shifts occur randomly, then the rank sums of their absolute values will be approximately equal. If the shift intensity in one of the directions outweighs, then the rank sum of absolute shifts in the opposite direction will be significantly lower than it would be with random changes.
In Figure 4, shifts in opposite directions are represented as different areas.
c
a) «Light front» predominates over «dark; front» in both the number of shifts and their intensity
n Initial indicators (body temperature before drug administration) Final indicators (body temperature after drug administration) Difference of indicators (d) Absolute difference Rank of the difference
1 39 37.6 -1.4 1.4 7
2 39.5 38.7 -0.8 0.8 5
33 38.6 38.7 4 39.1 38.5 5 440.1 38.6 0.1 -0.6 -1.5 0.1 0.6 1.5 1.5* 4 8
6 39.3 37.5 -1.8 1.8 9
7 38.9 38.8 -0.1 0.1 1.5
8 39.2 38 -1.2 1.2 6
9 39.8 39.8 0 0 0
10 38.8 39.3 0.5 0.5 3*
c) «Light front» is inferior to «dark front;» in the number
of shifts , bur the most inlense shifts belong to the "light front"
Figure 4. Types of the "light front" to "dark front" ratios; shifts in two different directionf [2(6]
Initially, it is arsumed that usual shift will shift in a more common disection, and unusual shift wül shift in a rarer ditection .
Let's state the hypotheses:
Ht: predominance of shifts between the initial and final indicatofsin one of the directions is unreliable (does not differ significantly from zero).
Alternative hypotheses:
Non-directional hypothesis:
H¡: predominance of shifts between the initial and final indicators in one ot the directions is reliable (significantly ngffetn from zero).
Dioectional hypothesis:
Hgf shift intensity m usual direction exceeds shift intern sitf in unusual direction (sigmficantly differs trom zero).
3.
Application of Wilcoxon T-test Make a list of pairs in any order. Calculate the difference between thr individual values in the second and the first measurements (before and after). Determine what wi ll b e considered the usual ehift and state the corresponding hypotheses. Convert the dfferences to absolute values and write them down m a separate column.
4. Rank the absolute values of the differences, assigning a lower rank to a lower value. Check the agreement of the received sum of ranks with the calculated one.
5. Note the ranks corresponding to shifts in unusual direction.
6. Calculate the sum of these ranks using the formula:, where is rank of shifts with a rarer sign.
7. Determine the critical values Tcr for a given volume of sampling n and the significance level a. If Texp < Tr, the null hypothesis is rejected, i. e. shift to the "usual" side in terms of intensity reliably predominates.
Limitations of Wilcoxon T-test
The minimum volume of sampling is 5.
Zero shifts are excluded and the number of observations is reduced by the number of these zero shifts. This limitation may not to be used by stating hypotheses that include no change: "Shift towards increasing values exceeds shift towards decreasing values and the tendency to keep them at the same level."
Example
Given data. A pharmaceutical company researches a new drug. A group of 10 volunteers suffering from disease was selected. They had their body temperature measured before and 30 minutes after taking the new drug. The data are presented in Table 7. It is required to conclude about the significance of a decrease in body temperature as a result of the drug administration [7]. Table 7. Given data for the problem
T3
o
-ja
__ ? !
!
1Ü21
«s
g
^ i E *
2 1,3 -s <u «U fi
* -s
U
i 1
ra —
.s o il
'c
c
g
1 g «s
• 8*1 ^ Ul 5
s? S
13 fi « c
1 2
3
4
5
6
7
8
9
10
39 39.5 338.(3
39.1 440.1 39.3 38.9
39.2 39.8 38.8
37.6
38.7
38.7
38.5
38.6 37.5
38.8 38
39.8 39.3
^ 3
S z
M fi*
¿3 .s
-1.4 -0.8 0.1 -0.6 -1.5 -1.8 -0.1 -1.2 0
0.5
«
-3 § J te
< T3
1.4 0.8 0.1 0.6
1.5 1.8 0.1 1.2 0
0.5
2
m
fi T3
1 * tf "o
7
5 1.5*
4
8 9
1.5
6 0
3*
* Unusual ranks are in bold italics. In this case, the atypical situation is that the temperature should decrease, but it increases.
Solution. Let's state the hypotheses:
H0: predominance of shifts between the initial and final indicators in one of the directions is unreliable (does not differ significantly from zero).
Hl: predominance of shifts between the initial and final indicators in one of the directions is reliable (significantly differs from ztro ).
Lets test ehe null hypothesis at the significance level a < 0.05.
The sum of unusual ranks is equal to the experimental value of the Wilcoxon T-test:
T = 1.5 + 3 = 4.5.
exp
Usual shift of the indicator (body temperature) is its decrease.
From the table of critical values presented in [7, 21, 25, 26], we can find.
Since the direction of the differences was not predicted in advance, we test the hypothesis for the two-sided test. For n = 9 (zero shift is excluded) and a = 0.05 T = 8.
Let's plot the axis of significance:
T = 4.5 T = 8
exp cr
Since Tcr > Texp, the null hypothesis is rejected. It can be stated that the reliability between the initi al and fin ail indicators in one of the directions is established.
Example
Given data:
In a group of students, training was conducted on the development of creative thinking. Tests were carried out before and after the training (stimulating effect should increase creativity, i. e. increase the score). The data are presented in Table 8. Is the hypothesis from the training providers confirmed that the training contributes to the development of creative thinking at the significance level a < 0.05?
Table 8. Given data for the problem and calculation results
n 1 Before training 1 Û After training i n Difference of indicators (d) Absolute difference Rank of the difference A C
1 2 ■x 19 20 1 Q 17 26 on -2 6 2 6 4.5 11 A IZ
3 4 c 18 15 OQ 20 18 2 3 1 2 3 1 4.5 6.5 o
5 6 n 29 21 30 25 1 4 n 1 4 n 2 8.5
7 8 û 21 18 28 19 OA 7 1 1 7 1 I 12 2
9 10 1 1 21 23 1 A 20 27 1 o -1 4 c 1 4 r 2 8.5 1 A
11 12 14 10 19 13 5 3 5 3 10 6.5
Solution. The first step in calculating the T-test is to subtract each individual "before" value from the "after" value. The calculation results in the form of the difference of indicators (d) are presented in Table 8. Two of the obtained differences are negative, and 10 of the differences are positive. Therefore, the usual direction is positive. Let's state the hypotheses:
H0: shift intensity in usual direction does not exceed shift intensity in unusual direction.
H shift intensity in usual direction exceeds shift intensity in unusual direction.
In the next step, all shifts, regardless of their sign, must; be ranked.
We check the correctness of the ranking.
£(R.) = 2 + 2 + 2 + 4.5 + 6.5 + 6.5 + 8.5 + 8.5 + 10 + 11 + 12 = 78
We check by the formula:
Z№)== 12 + »=78
Consequently, ranks are assigned correctly.
Note those shifts that are unusual, i. e. negative ones (in Table 8 they are marked in bold italics). The sum of the ranks of these unusual shifts is the experimental value of the T-test: T = 2 + 4.5 = 6.5.
exp
In this problem, the alternative hypothesis is directional because the direction of the differences is predicted in advance, so one-sided test should be used to test hypotheses. Foe n = 12 and a < 0.05, from the table of critical values presented in [7, 21, 25, 26], we can find Tr = 17.
Let's plot the axis of significance:
T = 6.5 T = 17
exp cr
Since Tcr > Texp, the null hypothesis is rejected. It can be seated that shift intensity in usual direction exceeds shift intensity in unusual direction, theredore, the initial assumption about she effectiveness oe the stimulating effect is confirmed.
Let the following test scores to be obtained for another group of subjects (Table 9). 6 differences aire negative, and 4 differences aire positivs, i. e. it is impossible to slate a hypothesis corresponding to the initial assumption. It is possible to staie a hypothesis assuming the insignificance or the indicator shift in the direction of decreasing:
H0: shift intensity towardc decreasing creativity does not exceed shift intensity towardi its mcrease.
H,: shift intensity towards decreasing creativity exceeds shift intensity towards its increase. Table 9. Given data for the problem and calculation results
1
2
3
4
5
6
7
8
9
10
Before training
19
23 18 15 25
2e 21 18
24 23
After training
17 20 20
18 17 15 20
19
20 28
Difference of indicators (d) -2 -3 2 3 -8 -6 -1 1
-4 5
Absolute difference
2 3 2
3 8 6 1 1
4
5
Rank of the difference 3.5 5.5 3.5 5.5 10 9
1.5 1.5
7
8
2
2
Let's rank the absolute differences and check the correctness of the ranking.
2CR.e = L5+ 1.5+ 3.5+ 3.5+ 5.5 + 5.5 + 7 + 8 + 9 + 10 = 55
By the formula we obtain:
N • (N+ 1) 10 • (12 + 1) rr l(R) = 2 = 2 = 55
Consequently, ranks are assigned correctly.
Note those shifts that are unusual, i. e. positive ones (in Table 9 they are marked in italics). Let's calculate the experimental value of T-test: T =3.5 + 5.5 + 1.5 + 8 = 18.5.
exp
n
Since the direction of the differences is predicted in advance, a one-sided test is used to test the hypotheses. For n = 10 and a < 0.05 Tr = 10. [7,21,25,26].
Let's plot the axis of significance
Since Te > T . the null hypothesis is accepled. It can be stated thae the intensity of the negative shift dotes not exceed the intensity of the positive shift.
Hie Wilcoxon T-test [37-49] is the most applicable for statistical processing in scientific researches. In [37], a study was con ducted to validatp the ecfX-tsrgeted qPCR protocot designed to dete ct all -viable P. aeruginosa bacteria and identify forms in sputum samples from patients with cystic fibrosis. The study involved 1151 P aeruginosa strains of various origins and 10 other strains in 88 cystic fibro-is sputum samples, 41 of which were positive and 47 were negative. Hie; statistical lignificance of the results was essessed using else of Wilcoxon T-test and Student t-test. The work [38] is aimed at improving diabetes care. The study is to assess the improvement of group care, which directly affects the quality of care for patients diagnosed with diabetes. Changes in laboratory values of A1c hemoglobin and LDL cholesterol were analyzed using the paired t-test and verified using the Wilcoxon T-test. The aim of work [39] was to conduct a study to pro-vide additional information on the health benefits of gluten-free foods and to test the hypothesis that both self-selected and recommended dieis are less nutritiour when gluten-free diets are followed. Diet differences were assessed using paired t-tests and ranks of the Wilcoxon T-test. Assessment of the amino acid profile in trauma patients and assessment of the nutritional mode effect and the addition of exogenous ALA-GLN dipeptide on plasma amino acid concentration were conducted in [40]. Differences between plasma amino acid concentrations on day 1 and day 6 weee assessed using; Student t-test, Mann-Whilniy U-test, and Wilcoxon T-test. In [41], f study of weight recovery after gastric bypass by increasing the consumption of high-calorie liquid and soft foods was carried out. Each patient with dysphagia was compared with 4 control patients based on the age, body mass index and time after surgery. Statistical analysis was performed using the Wil-coxon T-test. A study on the treatment of equine lamini-tis with and without acupuncture is reviewed in [42]. The study involved 14 horses with chronic laminitis. Lameness was statistically analyzed using the paired t-test and the sign ranked Wilcoxon T-test at p <0.05. An analysis of the effect of yerba mate daily consumption on serum lipids and body composition in overweight women is shown in [43]. 119 overweight 25 to 50 years old women were randomized into three groups: mate and regime (MD),
mate without regime (M), water and regime (AD). For 12 weeks, the M and MD groups were supplemented with mate, while the AD and MD groups followed a hypoca-loric regime. Anthropometric measurements and blood tests (total cholesterol, LDL cholesterol, HDL cholesterol, and triglycerides) were performed at the beginning and in the end of the study. Statistical analysis was performed using Student t-test and Wilcoxon T-test. Determination of the effect of a new phytosterol emulsion for nutritional supplements (1.5 g/day of phytosterol equivalent) on the concentration of low-density lipoprotein cholesterol was stu ctied i n [ 44]. Thirty-two healthy adults were randomized to receive placebo or treatment with a washout period followe d by placebo or treatment. Each phase lasted one month. Secondary endpoints related to cardiovascular health were assessed. Study management, including; screening, recruitment, monitoring, compliance and data collection, was carried out remotely (off-site clinical trial) ueing a new virtual instrument. The effect of phy-tosterol supplementation was assessed using Student t-test and Wilcoxon T-test. Changes in radiomic characteristics based on cone computed tomography (CBCT) during treetment and quantification of the potential relationship bet-ween CBCT-based delta radiomics characteristics and overall survival in locally advanced lung cancer patients have been reviewed [45]. 23 patients were identified and 658 radiomic characteristics were calculated for each of the 11 CBCT images in each patient. The choice of char-acteeistics was made on the basis of repeatability, contour uncertainty resistance, and lack of redundancy. Evaluation of the predictive ability for survival was carried out using the t-test and Wilcoxon T-test. The work [46] investigated the effect of smoking on periodontal disease. Comparison of 441 daily smokers with 441 non-smokers was performed using the of Wilcoxon T-test and t-test. The study [47] evaluated the reproducibility of 9 instruments for the analysis of differential expression in scRNA-seq data. Statistical analysis was performed using the t-test and Wilcoxon T-test. The study [48] was conducted from S eptember 2018 to August 2019 and included 101 athletes with disabilities (13 Paralympic disciplines) from Brasilia, Federal District. Food intake was assessed based on two or four inconsistent nutritional reviews over a 24-hour period, in which para-athletes reported all food, drinks, and supplements consumed in the previous 24 hours. Diet analysis was performed using the method of the National Cancer Institute. Comparisons between fellows and athletes not receiving a fellowship were made using the Student t-test for parametric variables and Wilcoxon T-test for nonparametric variables. Research [49] was focused on the recruitment and retention of women in Type 2 Diabetes Women Health Program, and to assess the program's initial effectiveness in improving well-being. A 12-week wellness event enrolled 70 middle-aged women with type 2 diabetes. Analyzes were performed using chi-squared tests, McNemar test, paired t-test, and Wilcoxon T-test.
Conclusion
The described nonparametric statistical tests are much less laborious than parametric ones, and for distributions far from normal (Gaussian), they are more efficient as well. It is completely insufficient for a current researcher to use one of the methods for statistical assessment of the differences between two groups of observations. In each case, a suitable test must be selected. This allows not only to in-
crease the efficiency of statistical processing and reduce its labor intensity, but also to get a clear idea of the comparison reliability of experimental results. The second part will review nonparametric tests for testing hypotheses of distribution type (Pearson's chi-squared test, Kolmogorov test) and nonparametric tests for testing hypotheses of sampling homogeneity (Pearson's chi-squared test for testing sampling homogeneity, Kolmogorov-Smirnov test).
REFERENCES
1. Fisher, R.A. (1992). Statistical methods for research workers. Chapter in a book: Breakthroughs in statistics. Springer Series in Statistics (Perspectives in Statistics). New York: Springer. 1992. https://doi.org/10.1007/978-1-4612-4380-96
2. Plokhinskiy, N.A. (1978). Mathematical methods in biology. Moscow: Moscow State University. 1978. (In Russian)
3. Glants, S. (1998). Medical and biological statistics. Moscow: Practice. 1998. (In Russian)
4. Lakin, G.F. (1990). Biometrics. Moscow: High School. 1990. (in Russian)
5. Rokitskiy, P.F. (1973). Biological statistics. Minsk: High School. 1973. (In Russian)
6. Snedecor, G.W. (1957). Statistical methods applied to experiments in agriculture and biology. US: The Iowa State college press. 1957.
7. Gubler, E.V., Genkin, A.A. (1973). Application of nonparamet-ric statistical criteria in biomedical research. Leningrad: Medicine. 1973. (In Russian)
8. Prokhorov, Yu.V. (1999). Probability and mathematical statistics: Encyclopedia. Moscow: The Great Russian Encyclopedia. 1999. (In Russian)
9. Hollander, M., Wolfe, D.A. (1999). Nonparametric statistical methods. New York: Wiley-Interscience. 1999.
10. Hettmansperger, T.P. (1991). Statistical inference based on ranks. New York: Krieger Pub Co. 1991.
11. Kendall, M.G. (1962). Rank correlation methods. Chicago: Hafner Publishing Company. 1962.
12. Gibbons, J., Chakraborti, S. (2003). Nonparametric statistical inference. Fourth Edition, Revised and Expanded. New York: Marcel Dekker. 2003.
13. Lehmann, E.L. (1975). Nonparametric statistical methods based on ranks. San Francisco, Düsseldorf etc.: McGraw-Hill. 1975.
14. Puri, M.L., Sen, P.K. (1985). Nonparametric methods in general linear models. New York etc.: Wiley. 1985.
15. Pratt, J.W., Gibbons, J.D. (1981). Concepts of nonpara-metric theory. New York: Springer-Verlag. 1981. https://doi. org/10.1007/978-1-4612-5931-2
16. Sidak, Z., Sen, P.K., Hajek, J. (1999). Theory of rank tests. US: Academic Press. 1999.
17. Tyurin, Yu.N., Makarov, A.A. (1998). Statistical analysis of data on a computer. Moscow: Infra-M. 1998. (In Russian)
18. Tyurin, Yu.N., Shmerling, D.S. (2004). Nonparametric methods of statistics. Sociology: Methodology, Methods, Mathematical models, 18, 154-166. (In Russian)
19. Siegel, S. (1957). Nonparametric statistics. The American Statistician, 11 (3), 13-19. https://doi.org/10.1080/01621459 .1957.10501392
20. Mann, H.B., Whitney, D.R. (1947). On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics, 18, 50-60. https://doi. org/10.1214/aoms/1177730491
21. Rosenbaum, S. (1954). Tables for a nonparametric test of location. Annals of Mathematical Statistics, 25 (1), 146-150. https://doi.org/10.1214/aoms/1177728854
22. Wilcoxon, F. (1945). Individual Comparisons by ranking methods. Biometrics Bulletin, 1(6), 80-83. http://www.jstor.org/sta-ble/3001968.
23. Wilcoxin, F. (1947). Probability tables for individual comparisons by ranking methods. Biometrics, 3(3), 119-122. https://doi. org/10.2307/3001946
24. Wilcoxon, F. (1946). Individual comparisons of grouped data by ranking methods. Journal of Economic Entomology, 39, 269. https://doi.org/10.1093/jee/39.2.269
25. Stepanov, V.G. (2019). Application of nonparametric statistical methods in agricultural biology and veterinary medicine research. St-Petersburg: Lan. 2019. (In Russian)
26. Edelbaeva, N.A., Lebedinskaya, O.G., Kovanova, E.S., Teneto-va, E.P., Timofeev, A.G. (2019). Fundamentals of nonparametric statistics. Moscow: YUNITY-DANA. 2019. (In Russian)
27. Kuz'mina, E.V., P'yankova, N.G., Tret'yakova, N.V., Botsoe-va, A.V. (2020). Using data analysis methodology to foster professional competencies in business informaticians. European Journal of Contemporary Education, 9(1), 54-66. https://doi. org/10.13187/ejced.2020.1.54
28. Podrihalo, O., Savina, S., Podrigalo, L., lermakov, S., Jagietto, W., Rydzik, t. et al. (2020). Influence of health-related fitness on the morphofunctional condition of second mature aged women. International Journal of Environmental Research and Public Health, 17(22), Article 8465, 1-9. https://doi.org/10.3390/ ijerph17228465
29. Kotlyarova, I., Clmvasliova, A. (2020). Educational imitation of basic job function using the knowledge of English among technical Major students. International Journal of Instruction, 14(1), 303-324. https://doi.org/10.29333/IJI.2021.14118A
30. Mueller, S. K., Nocera, A. L., Dillon, S. T., Libermann, T. A., Wendler, O., Bleier, B. S. (2019). Tissue and exosomal serine protease inhibitors are significantly overexpressed in chronic rhinosinus-itis with nasal polyps. American Journal of Rhinology and Allergy, 33(4), 359-368. https://doi.org/10.1177/1945892419831108
31. Gilic, F., Schultz, K., Sempowski, I., Blagojevic, A. (2019). "Nightmares-family medicine" course is an effective acute care teaching tool for family medicine residents. Simulation in Healthcare, 14(3), 157-162. https://doi.org/10.1097/SIH.0000000000000355
32. Zabetian, H., Sadeghi, F., Falah, A., Kalani, N. (2018). Studying the Effect of Intravenous Injections of Ketorolac (IVIK) on analgesia control before and after using tourniquet in orthopedic surgery of femur and tibia by general anesthesia. Journal of Research in Medical and Dental Science, 6(2), 227-232. https://doi.org/10.24896/jrmds.20186235
33. Spielberger, J., Heid, F., Schmidtmann, I., Drees, P., Betz, U., Schwaderlapp, W. et al. (2021). Patient-centered perioperative vigilance: perioperative process quality, effectiveness of pain treatment and mobilization progress after implementation of a treatment bundle for total knee endoprosthesis. Anaesthe-sist, 70(3), 213-222. https://doi.org/10.1007/s00101-020-00874-8
34. Regnery, S., Behl, N. G. R., Platt, T., Weinfurtner, N., Windisch, P., Deike-Hofmann, K. et al. (2020). Ultra-high-field sodium MRI as biomarker for tumor extent, grade and IDH mutation status in glioma patients. NeuroImage: Clinical, 28, Article 102427. https://doi.org/10.1016/j.nicl.2020.102427
35. McDonald, J.H. (2014). G-test of goodness-of-fit. Baltimore, Maryland: Sparky House Publishing. 2014.
36. McDonald, J.H. (2014). Small numbers in chi-square and G-tests. Baltimore, Maryland: Sparky House Publishing. 2014.
37. Mangiaterra, G., Amiri, M., Di Cesare, A., Pasquaroli, S., Manso, E., Cirilli, N., Citterio, B. et al. (2018). Detection of viable but non-culturable Pseudomonas aeruginosa in cystic fibrosis by qP-CR: a validation study. BMC Infectious Diseases, 18, Article 701. https://doi.org/10.1186/s12879-018-3612-9
38. Bilello, L.A., Scuderi, C., Haddad, I.C.J., Smotherman, C., Shahady, E. (2018). Practice transformation: using team-based care training to improve diabetes outcomes. Journal of primary care and community health, 9. https://doi. org/10.1177/2150132718817952
39. Taetzsch, A., Das, S. K., Brown, C., Krauss, A., Silver, R. E., Roberts, S. B. (2018). Are gluten-free diets more nutritious? An
evaluation of self-selected and recommended gluten-free and gluten-containing dietary patterns. Nutrients, 10(12), Article 1881. https://doi.org/10.3390/nu10121881
40. Raurich, J. M., Llompart-Pou, J. A., Garcia-de-Lorenzo, A., Buno Soto, A., Marse, P., Frontera, G. et al. (2018). Effect of the route of nutrition and l-alanyl-l-glutamine supplementation in amino acids' concentration in trauma patients. European Journal of Trauma and Emergency Surgery, 44(6), 869-876. https://doi. org/10.1007/s00068-017-0851-1
41. Runge, T.M., Jirapinyo, P., Chan, W.W., Thompson, C.C. (2019). Dysphagia predicts greater weight regain after Roux-en-Y gastric bypass: a longitudinal case-matched study. Surgery for obesity and related diseases, 15(12), 2045-2051. https://doi. org/10.1016/j.soard.2019.06.041
42. Lee, D., May, K., Faramarzi, B. (2019). Comparison of first and second acupuncture treatments in horses with chronic laminitis. Iranian Journal of Veterinary Research, 20(1), 9-12.
43. Avena, V., Messina, D., Corte, C., Mussi, J., Saez, A., Boarelli, P. et al. (2019). Association between consumption of yerba mate and lipid profile in overweight women. Nutricion hospitalaria, 36(6), 1300-1306. https://doi.org/10.20960/nh.02599
44. Reaver, A., Hewlings, S., Westerman, K., Blander, G., Schmeller, T., Heer, M. et al. (2019). A randomized, placebo-controlled, double-blind crossover study to assess a unique phytoster-ol ester formulation in lowering LDL cholesterol utilizing a novel
virtual tracking tool. Nutrients, 11(9), Article 2108. https://doi. org/10.3390/nu11092108
45. Shi, L., Rong, Y., Daly, M., Dyer, B., Benedict, S., Qiu, J. et al. (2020). Cone-beam computed tomography-based delta-radiomics for early response assessment in radiotherapy for locally advanced lung cancer. Physics in Medicine and Biology, 65(1), Article 015009. https://doi.org/10.1088/1361-6560/ab3247
46. Shauly-Aharonov, M. (2020). An exact test with high power and robustness to unmeasured confounding effects. Statistics in Medicine, 39(8), 1041-1053. https://doi.org/10.1002/sim.8460
47. Mou, T., Deng, W.J., Gu, F.Y., Pawitan, Y., Vu, T.N. (2020). Re-producibility of methods to detect differentially expressed genes from single-cell RNA sequencing. Frontiers in Genetics, 10, Article 1331. https://doi.org/10.3389/fgene.2019.01331
48. Sasaki, C.A.L., da Costa, T.H.M. (2021). Micronutrient deficiency in the diets of para-athletes participating in a sports scholarship program. Nutrition, 81, Article 110992. https://doi. org/10.1016/j.nut.2020.110992
49. Anderson, D., Sturt, J., McDonald, N., White, C., Porter-Steele, J., Rogers, R. et al. (2021). International feasibility study for the women's wellness with type 2 diabetes programme (WWDP): An eHealth enabled 12-week intervention programme for midlife women with type 2 diabetes. Diabetes Research and Clinical Practice, 171, Article 108541. https://doi.org/10.1016/j.di-abres.2020.108541
AUTHOR INFORMATION
Marina A. Nikitina — candidate of technical sciences, docent, leading scientific worker, the Head of the Direction of Information Technologies of the Center of Economic and Analytical Research and Information Technologies, V. M. Gorbatov Federal Research Center for Food Systems of Russian Academy of Sciences. 26, Talalikhina, 109316, Moscow, Russia. Tel: +7-495-676-95-11 extension 297, E-mail: m.nikitina@fncps.ru ORCID: https://orcid.org/0000-0002-8313-4105 * corresponding author
Irina M. Chernukha — doctor of technical sciences, professor, Academician of the Russian Academy of Sciences, Head of the Department for Coordination of Initiative and International Projects, V. M. Gorbatov Federal Research Center for Food Systems of Russian Academy of Sciences. 26, Talalikhina, 109316, Moscow, Russia. Tel: +7-495-676-95-11 extension 109, E-mail: imcher@inbox.ru ORCID: https://orcid.org/0000-0003-4298-0927
All authors bear responsibility for the work and presented data.
All authors made an equal contribution to the work.
The authors were equally involved in writing the manuscript and bear the equal responsibility for plagiarism. The authors declare no conflict of interest.
