CC BY-NC-ND
12
Psychology. Journal of the Higher School of Economics. 2023. Vol. 20. N 1. P. 31-39. Психология. Журнал Высшей школы экономики. 2023. Т. 20. № 1. С. 31-39. DOI: 10.17323/1813-8918-2023-1-31-39
DETERMINANTS OF THE ONLINE PROSOCIAL BEHAVIOR DURING THE COVID-19 PANDEMIC
M.A. BULTSEVAa
a HSE University, 20 Myasnitskaya Str, Moscow, 101000, Russian Federation
Детерминанты онлайн-просоциального поведения во время пандемии COVID-19
М.А. Бульцева"
' Национальный исследовательский университет «Высшая школа экономики», 101000, Россия, Москва, ул. Мясницкая, д. 20
Abstract
The COVID-19 pandemic has a significant negative impact on people's life and behavior. This happens due to circumstances beyond the control of people, which lead to social isolation, increase in fear, anxiety, and other negative feelings and states. However, these negative feelings and states may have not only negative consequences, but also trigger prosocial behavior as a coping strategy. Prosocial behavior online is a relatively new domain of prosocial behavior developed due to an increase in ICT use in recent decades. Prosocial behavior online is more safe and less costly than offline one, and thus can be more relevant in times of the pandemic. This research was aimed at identifying the relationship between negative emotional states actualized by the COVID-19 pandemic (fear of COVID-19, stress, anxiety, depres-
Резюме
Пандемия COVID-19 оказывает негативное влияние на жизнь людей. Невозможность контролировать ситуацию и разного рода ограничения приводят к социальной изоляции и росту выраженности страха, тревоги и других негативных эмоциональных состояний. Подобные состояния несут в себе не только опасность негативных последствий, но и потенциал для стимулирования просоциального поведения как копинговой стратегии. В последние десятилетия в связи с развитием информационно-коммуникационных технологий особый интерес вызывает тема просоциального поведения онлайн. Предположение о позитивной связи негативных эмоциональных состояний и вовлеченности в просоциальное поведение кажется особенно актуальным именно для онлайн-контекста. Дело в том, что просоциальное поведение онлайн является более безопасным и менее ресурсозат-ратным, чем просоциальные действия в реальной жизни, что может быть важно в условиях
The study was supported by the Russian Science Foundation, project N 19-18-00169, https://rscf.ru/project/19-18-00169
Исследование выполнено при поддержке Российского научного фонда, проект № 19-18-00169, https://rscf.ru/project/19-18-00169
sion) and prosocial behavior online. Cross-sectional study was conducted on a Russian sample (N=215) with the help of such instruments as Scale of Prosocial Behavior in an Online Context, Fear of COVID-19 scale and DASS-21. Regression analysis has shown that fear and anxiety contributed to prosocial behavior online, while depression inhibited it. The relationship between stress level and prosocial behavior online was not statistically significant. The results are discussed considering arousal potential of different negative emotional states.
пандемии. Поэтому данное исследование нацелено на выявление взаимосвязи негативных эмоциональных состояний, актуализированных пандемией (страха перед коронавирусным заболеванием, стресса, тревоги и депрессии) с просо-циальным поведением онлайн. Исследование было реализовано с применением кросс-секционного дизайна на выборке из 215 россиян. Были использованы Шкала просоциального поведения в онлайн-контексте, шкала страха перед коронавирусным заболеванием и русскоязычная версия теста DASS-21. По итогам регрессионного анализа было выявлено, что страх перед коронавирусным заболеванием и тревога стимулируют онлайн-просоциальное поведение, а депрессивное состояние препятствует ему. Уровень стресса оказался не связан с просоциальным поведением онлайн. При обсуждении результатов принимается во внимание «энергетический» потенциал различных эмоциональных состояний.
Keywords: prosocial behavior, online context, COVID-19, fear of coronavirus, stress, anxiety, depression.
Maria А. Bultseva — Junior Research Fellow, Center for Socio-Cultural Research, HSE University, PhD in Psychology.
Research Area: creativity, prosocial behavior, intercultural learning, psychological effects of digitalization. E-mail: [email protected]
Ключевые слова: просоциальное поведение, онлайн-контекст, COVID-19, страх перед коро-навирусом, стресс, тревога, депрессия
Бульцева Мария Александровна — младший научный сотрудник, Центр социокультурных исследований, Национальный исследовательский университет «Высшая школа экономики», кандидат психологических наук. Сфера научных интересов: креативность, про-социальное поведение, межкультурное обучение, психологические эффекты цифровизации. Контакты: [email protected]
Prosocial behavior covers a diverse set of acts which benefit other people and, thus, is important for individuals and society (Penner et al., 2005). Prosocial behavior has positive outcomes not only for the recipients of help, but for the helper as well, increasing his/her positive affect, well-being, sense of meaning in life and involvement in social relationships (Baumsteiger, 2017).
The opportunities people have to help others are rapidly evolving during the digital era. Online prosocial behavior is becoming more popular, and often is more pronounced than prosocial behavior in real life settings (Jiang et al., 2017). Online environments reduce barriers in helping others, and make prosocial behavior more noticeable and public, and less costly or dangerous (Sproull, 2011). It also gives people opportunities to create a sense of belonging to a desirable online community and to fulfill their social needs (Armstrong-Carter & Telzer, 2021).
The COVID-19 pandemic made people more engaged in different types of online communication (Nguyen et al., 2020). Scientists underline the importance of online prosocial behavior in this period of crisis (Miao et al., 2021). The fast spread of the coronavirus disease (COVID-19) had a drastic impact on physical and mental health of people. The pandemic created the fear of getting sick and made death salience more pronounced, and increased levels of stress, depression and anxiety in different countries (Luo et al., 2020; Passavanti et al., 2021), including Russia (Zinchenko et al., 2021). Lockdowns and physical distancing led to disruption of social interactions, and consequently, to an increase in loneliness and social isolation (Smith & Lim, 2020). This could significantly limit people's opportunities to help others and stimulate concerns centered on one's own safety. However, recent studies have demonstrated that people are keen to cooperate and behave prosocially during the pandemic despite all the limitations (Haller et al., 2022; Hellmann et al., 2021).
Negative feelings and emotional states, elicited by the pandemic, could serve as a trigger for prosocial behavior. First, people who experience negative feelings use helping behavior as a coping strategy to change their mood (van Kleef & Lelieveld, 2022). Indeed, helping others was proven to have an immediate positive influence on a helper's mood during the pandemic (Varma et al., 2020). Next, negative feelings associated with shared suffering may make people more attentive to the needs of others. For instance, cues associated with the COVID-19 pandemic elicited negative emotions which may guide prosocial behavior through empathy (Li, 2021). All these can be especially relevant for the online prosocial behavior as it is more available and safer for people during the pandemic.
To conclude, the online prosocial behavior may be one of the strategies people use to cope with the sufferings they experience during the pandemic. Specifically, people experience fear associated with the disease, high stress, depression and anxiety. These negative feelings and emotional states may stimulate people to engage in the online prosocial behavior more often in order to alleviate their state. Therefore, the following hypotheses can be formulated: high levels of fear of COVID-19 (a), depression (b), anxiety (c) and stress (d) positively relate to the online prosocial behavior.
Method
Sample
The sample of the study was collected during the first quarter of 2022 when there was a significant increase in the number of infected people in Russia (the highest wave from the beginning of the pandemic). In total, 215 respondents constituted the sample (aged 18 to 58, mean age M = 22.98 years, a = 6.21) from 51 settlements in different regions of the Russian Federation. The majority of the sample are Russians (91.1%), women (69.8%), non-religious people (68.8%), people with higher education or students (84.6%), and employed people (55.8%).
Design and Procedure
A quantitative cross-sectional study was conducted online on the anketolog.ru platform. All the instruments were in the Russian language. Respondents started to fill in the questionnaire by reading the informed consent form and displaying the agreement to participate in the study. Next, they answered questions regarding the feelings they had during the previous few weeks. After that they indicated the frequency of engagement in different prosocial activities online. Finally, the respondents specified information about their age, gender and other socio-demographic characteristics. On average, respondents needed 12 minutes to fill in the questionnaire.
Measurements
The online prosocial behavior was measured with the help of the Scale of Prosocial Behavior in an Online Context (Efremova & Bultseva, 2020), composed by 13 items measured on a 5-point scale from 1 — "never" to 5 — "very often". The respondents were asked to indicate how often they have been involved in different prosocial activities on the Internet during the recent few weeks, e.g. "I provided advice to someone on the Internet for free on issues that I know well". The higher mean score for 13 items represents a greater involvement.
The Fear of COVID-19 Scale (Ahorsu et al., 2022), validated on a Russian sample by Gritsenko et al. (2020), was used to measure the intensity of fear the respondents had regarding the pandemic. It included 7 items measured on a 5-point scale from 1 — "strongly disagree" to 5 — "strongly agree". The respondents were asked to indicate to what degree they agreed with the statements regarding their feelings during the last few weeks of the COVID-19 pandemic, e.g. "It makes me uncomfortable to think about coronavirus-19". The higher the sum of the 7 items was, the more pronounced fear was reported.
Depression, anxiety and stress levels of the respondents were measured with the help of DASS-21, a short version of the Depression, Anxiety, Stress Scale (Lovibond & Lovibond, 1995) adapted and validated on the Russian sample by Zolotareva (2021). It is comprised of 21 items, 7 per each subscale, on a 4-point scale from 0 — "never" to 3 — "always". The scale can be used in both clinical and non-clinical settings (Jiang et al., 2020). Respondents were asked to indicate how often they experienced the states and feelings described in the statements in the recent few weeks, e.g. "I couldn't seem to experience any positive feelings at all" (for depression), "I was worried about situations in which I might panic and make a fool of myself" (for anxiety), "I tended to over-react to situations" (for stress). The higher score on each subscale represents greater levels of psychological distress.
Data processing
First, internal consistency of the instruments was stated through reliability analysis (Cronbach's alpha test) and analysis of descriptive statistics was performed. Next,
in order to test the hypotheses, regression analysis was applied. In the regression analysis, the prosocial behavior in the online context represented a dependent variable, while COVID-19 fear, depression, anxiety and stress levels were independent variables. Statistical analysis was performed in the SPSS 22.0 program.
Results
The first stage of statistical analysis included descriptive statistics calculations and correlation analysis (Table 1). The respondents demonstrated the medium intensity of COVID-19 fear. Compared to norms for the Russian population (Gritsenko et al., 2020), 58.1% of respondents have a low level of fear (score 14 and below), 7.9% have a medium level of fear (score 15-18) and 34% of respondents have a high level of fear (score 19 and higher). As for DASS-21 subscales, compared to the international norms (Lovibond & Lovibond, 1995), the majority of respondents were below the normal threshold (Table 2).
Table 1
Descriptive Statistics, Reliability and Correlations
Variables Mean (SD) Reliability (a) 1 2 3 4 Age Gender
1. Online prosocial behavior 2.89 (0.81) 0.89 - 0.02 -0.05
2. Covid-19 Fear 14.64 (7.43) 0.94 0.35** - 0.12 -0.12
3. DASS-21 — depression ) 0.81 0.02 0.39** - -0.03 0.01
4. DASS-21 — anxiety 5.43 (5.10) 0.8 0.36** 0.62** 0.72** - -0.04 -0.03
5. DASS-21 — stress 9.09 (6.26) 0.83 0.23** 0.48** 0.73** 0.77** 0.03 0.03
Note. In accordance with the recommendations of Lovibond & Lovibond (1995), in order to get the DASS-21 scores, the sum of the 7 items per each subscale was multiplied by 2. ** — p < .01.
Table 2
Percentage of Respondents Experiencing Different Levels of Depression, Stress and Anxiety during the COVID-19 Pandemic
Level/State Depression Anxiety Stress
Normal 73.5% 72.6% 85.6%
Mild 11.6% 6% 11.1%
Moderate 11.6% 17.2% 1.3%
Severe 2.8% 3.3% 0.4%
Extremely severe 0.5% 0.9% 0.5%
Table 3
The Relationship between Negative Feelings and States and Online Prosocial Behavior
Negative feelings & states Fear Depression Anxiety Stress
Coefficients (fi) 0.16* -0.49** 0.56** 0.07
R2 0.27
F 19.09**
Cohen's f2 0.37
Correlation analysis demonstrated the existence of a positive relationship between negative feelings associated with the COVID-19 pandemic (except for depression) and the online prosocial behavior. Gender and age didn't have any significant relationship with the online prosocial behavior, so they were excluded from further analysis.
The results of a regression analysis showed that states associated with the COVID-19 pandemic, indeed, have an effect on the online prosocial behavior. Specifically, anxiety and COVID-19 fear contribute to the online prosocial behavior, while depression limits it. This result supports the hypothesis of this study regarding fear (a) and anxiety (c), but rejects the hypothesis regarding depression (b) and stress (d).
Discussion
Contrary to the hypotheses, negative emotional states were differently related to the online prosocial behavior. The results on fear and anxiety support the idea that negative feelings may lead to prosocial behavior. Prosocial behavior can be a response to the experience of common suffering during traumatic events such as pandemic, war or natural disaster (Haller et al., 2022; Hartman & Morse, 2020; Kaniasty & Norris, 1995). On the other hand, based on the results, depression negatively relates to the online prosocial behavior, while stress is not related to it. This raises the question: why are particular negative emotional states important for the online prosocial behavior?
People experience these emotional states differently. Fear and anxiety are associated with higher levels of arousal (Pizzie & Kraemer, 2021) motivating people to act. Based on the interpretations about arousal, people are involved in prosocial behavior with either egoistic motivation of improving their emotional state or altruistic motivation of empathy (Batson et al., 1991; Penner et al., 2005).
Excessive arousal, however, reduces the empathy towards others and, consequently, limits prosocial behavior (Eisenberg, 2002). High levels of fear and anxiety had a negative effect on helping in China (Ye et al., 2020). At the same time, feelings should be intense enough to make people involved in a prosocial behavior. For example, people experiencing acute anxiety were more engaged in prosocial behavior (Vieira et al., 2020). Generally, that is in line with the notion of the inverted u-curve relationship between arousal and performance (Larson & Moses, 2014). In this particular study, the levels of fear and anxiety were medium, so it could be a bearable "optimum" for the majority of the respondents.
Depression, on the contrary, is often described as a state with a low level of emotional arousal (Benning & Ait Oumeziane, 2017). Previous studies also found that depression makes people less involved in interpersonal relations and less keen for prosocial acts (Wentzel et al., 2007). So, people experiencing depression may have too few available "emotional resources" to engage in prosocial behavior.
Finally, stress can be differently related to prosocial behavior (Nitschke et al., 2022). The relationship depends on the preferred coping strategy and cultural norms and values (Haller et al., 2022). Probably, differences in attitudes towards stress and coping strategies of the respondents were the reason why, in this study, stress is not related to the online prosocial behavior.
To conclude, the study obtained novel results specifying the relationship between specific negative emotional states and prosocial behavior online. However, it has several limitations as well.
This study was based on the idea that people use prosocial behavior to cope with negative emotions or because they share the suffering of others. However, specific motivation for prosocial behavior, as well as preferred coping strategies, were not measured. A more elaborate study, taking them into account, may refine the findings.
Next, not only emotional states can influence prosocial behavior, but reversed relationships are also possible. Engagement in prosocial behavior can contribute to people's mental health and well-being by increasing the positive effect and reducing stress, anxiety, and depression (Varma et al., 2022). A diary study or an experimental study with several stages of measurements would allow us to look at the relationship between emotional states and prosocial behavior in a complex.
Finally, the online prosocial behavior only was considered in this study. It differs from prosocial behavior in real life settings, and even more than that, the pandemic widens digital inequalities and makes the less tech-savvy (e.g., elder people, poor people without an access to the Internet etc.) excluded from communication and helping (Nguen et al., 2020). So additional research is needed on prosocial behavior in the other context and on specific population groups.
References
Ahorsu, D. K., Lin, C. Y., Imani, V., Saffari, M., Griffiths, M. D., & Pakpour, A. H. (2022). The fear of COVID-19 scale: development and initial validation. International Journal of Mental Health and Addiction, 20(3), 1537-1545. https://doi.org/10.1007/s11469-020-00270-8 Armstrong-Carter, E., & Telzer, E. H. (2021). Advancing measurement and research on youths' prosocial behavior in the digital age. Child Development Perspectives, 15(1), 31-36. https://doi.org/10.1111/cdep.12396 Batson, C. D., Batson, J. G., Slingsby, J. K., Harrell, K. L., Peekna, H. M., & Todd, R. M. (1991). Empathic joy and the empathy-altruism hypothesis. Journal of Personality and Social Psychology, 61(3), 413-426. https://doi.org/10.1037/0022-3514.6L3.413 Baumsteiger, R. (2017). Looking forward to helping: The effects of prospection on prosocial intentions and behavior. Journal of Applied Social Psychology, 47(9), 505-514. https://doi.org/10.1111/jasp.12456
Benning, S. D., & Ait Oumeziane, B. (2017). Reduced positive emotion and underarousal are uniquely associated with subclinical depression symptoms: Evidence from psychophysiology, self report, and symptom clusters. Psychophysiology, 54(7), 1010-1030. https://doi.org/10.1111/psyp.12853 Efremova, M. V., & Bultseva, M. A. (2020). The relationship between individual values and prosocial behavior in an online and offline contexts. Sotsial'naya Psikhologiya i Obshchestvo [Social Psychology and Society], 11(1), 107-126. https://doi.org/10.17759/sps.2020110107 (in Russian) Eisenberg, N. (2002). Distinctions among various modes of empathy-related reactions: A matter of importance in humans. Behavioral and Brain Sciences, 25(1), 33-34. https://doi.org/10.1017/S0140525X02350015 Gritsenko, V. V., Reznik, A. D., Konstantinov, V. V., Marinova, T. Y., Khamenka, N. V., & Isralowitz, R. (2020). Fear of Coronavirus Disease (COVID-19) and basic personality beliefs. Klinisheskaya i Spetsial'naya Psikhologiya [Clinical Psychology and Special Education], 9(2), 99-118. https://doi.org/10.17759/cpse.2020090205 (in Russian) Haller, E., Lubenko, J., Presti, G., Squatrito, V., Constantinou, M., Nicolaou, C., Papacostas, S., Aydin, G., Chong, Y. Y., Chien, W. T., Cheng, H. Y., Ruiz, F. J., Garcia-Martin, M. B., Obando-Posada, D.
P., Segura-Vargas, M. A., Vasiliou, V. S., McHugh, L., Höfer, S., Baban, A., Dias Neto, D.....Gloster,
A. T. (2021). To help or not to help? Prosocial behavior, its association with well-being, and predictors of prosocial behavior during the coronavirus disease pandemic. Frontiers in Psychology, 12, Article 775032. https://doi.org/10.3389/fpsyg.2021.775032 Hartman, A. C., & Morse, B. S. (2020). Violence, empathy and altruism: evidence from the Ivorian refugee crisis in Liberia. British Journal of Political Science, 50, 731-755. https://doi.org/10.1017/S0007123417000655 Hellmann, D. M., Dorrough, A. R., & Glöckner, A. (2021). Prosocial behavior during the COVID-19 pandemic in Germany. The role of responsibility and vulnerability. Heliyon, 7(9), Article e08041. https://doi.org/10.1016Zj.heliyon.2021.e08041 Jiang, H., Chen, G., & Wang, T. (2017). Relationship between belief in a just world and Internet altruistic behavior in a sample of Chinese undergraduates: Multiple mediating roles of gratitude and self-esteem. Personality and Individual Differences, 104, 493-498. https://doi.org/10.1016/j.paid.2016.09.005 Jiang, L. C., Yan, Y. J., Jin, Z. S., Hu, M. L., Wang, L., Song, Y., Li, N.-N., Su, J., Wu, D.-X., & Xiao, T. (2020). The depression anxiety stress Scale-21 in Chinese hospital workers: reliability, latent structure, and measurement invariance across genders. Frontiers in Psychology, 11, Article 247. https://doi.org/10.3389/fpsyg.2020.00247 Kaniasty, K., & Norris, F. H. (1995). In search of altruistic community: patterns of social support mobilization following Hurricane Hugo. American Journal of Community Psychology, 23, 447-477. https://doi.org/10.1007/BF02506964 Larson, A., & Moses, T. (2014). Examining the link between stress events and prosocial behavior in
adolescents. Youth & Society, 49(6), 779-804. https://doi.org/10.1177/0044118x14563049 Li, D. (2021, December). The cues of COVID-19: Negative emotions and empathy guide prosocial behavior. In 2021 4th International Conference on Humanities Education and Social Sciences (ICHESS 2021) (pp. 1968-1975). Atlantis Press. https://doi.org/10.2991/assehr.k.211220.336 Lovibond, S. H., & Lovibond, P. F. (1995). Manual for the Depression Anxiety & Stress Scales (2nd ed.).
Sydney: Psychology Foundation. https://maic.qld.gov.au/wp-content/uploads/2016/07/DASS-21.pdf Luo, M., Guo, L., Yu, M., & Wang, H. (2020). The psychological and mental impact of coronavirus disease 2019 (COVID-19) on medical staff and general public - A systematic review and meta-analysis. Psychiatry Research, 291, Article 113190. https://doi.org/10.1016/j.psychres.2020.113190
Miao, Q., Schwarz, S., & Schwarz, G. (2021). Responding to COVID-19: Community volunteerism and coproduction in China. World Development, 137, Article 105128. https://doi.org/10.1016/j.worlddev.2020.105128 Nguyen, M. H., Gruber, J., Fuchs, J., Marler, W., Hunsaker, A., & Hargittai, E. (2020). Changes in digital communication during the COVID-19 global pandemic: Implications for digital inequality and future research. Social Media + Society, 6(3), Article 2056305120948255. https://doi.org/10.1177/2056305120948255 Nitschke, J. P., Forbes, P., & Lamm, C. (2022, July 15). Does stress make us more - or less - prosocial? A systematic review and meta-analysis of the effects of acute stress on prosocial behaviours using economic games. PsyArXiv Preprints. https://doi.org/10.31234/osf.io/2zjtd Passavanti, M., Argentieri, A., Barbieri, D. M., Lou, B., Wijayaratna, K., Mirhosseini, A. S. F., Wang, F., Naseri, S., Qamhia, I., Tangeres, M., & Ho, C.-H. (2021). The psychological impact of COVID-19 and restrictive measures in the world. Journal of Affective Disorders, 283, 36-51. https://doi.org/10.1016/jjad.2021.01.020 Penner, L. A., Dovidio, J. F., Piliavin, J. A., & Schroeder, D. A. (2005). Prosocial behavior: Multilevel perspectives. Annual Review of Psychology, 56, 365-392. https://doi.org/10.1146/annurev.psych.56.091103.070141 Pizzie, R. G., & Kraemer, D. J. (2021). The association between emotion regulation, physiological arousal, and performance in math anxiety. Frontiers in Psychology, 12. https://doi.org/639448.10.3389/fpsyg.2021.639448 Smith, B. J., & Lim, M. H. (2020). How the COVID-19 pandemic is focusing attention on loneliness and
social isolation. Public Health Research Practice, 30, 2-5. https://doi.org/10.17061/phrp3022008 Sproull, L. (2011). Prosocial behavior on the net. Daedalus, 140(4), 140-153.
https://doi.org/10.1162/DAED_a_00120 Van Kleef, G. A., & Lelieveld, G. J. (2022). Moving the self and others to do good: the emotional underpinnings of prosocial behavior. Current Opinion in Psychology, 44, 80-88. https://doi.org/10.1016/jxopsyc.2021.08.029 Varma, M. M., Chen, D., Lin, X., Aknin, L. B., & Hu, X. (2022). Prosocial behavior promotes positive emotion during the COVID-19 pandemic. Emotion. Advance online publication. https://doi.org/10.1037/emo0001077 Vieira, J., Pierzchajlo, S., Jangard, S., Marsh, A. & Olsson, A. (2020). Perceived threat and acute anxiety predict increased everyday altruism during the COVID-19 pandemic. PsyAchXiv. https://doi.org/10.31234/osf.io/n3t5c Wentzel, K. R., Filisetti, L., & Looney, L. (2007). Adolescent prosocial behavior: The role of self processes and contextual cues. Child Development, 78(3), 895-910. https://doi.org/10.1111/j.1467-8624.2007.01039.x Ye, Y., Long, T., Liu, C. & Xu, D. (2020). The effect of emotion on prosocial tendency: The moderating effect of epidemic severity under the outbreak of COVID-19. Frontiers in Psychology, 11, Article 3550. https://doi.org/10.3389/fpsyg.2020.588701 Zinchenko, Y. P., Shaigerova, L. A., Almazova, O. V., Shilko, R. S., Vakhantseva, O. V., Dolgikh, A. G., Veraksa, A. N., & Kalimullin, A. M. (2021). The spread of COVID-19 in Russia: immediate impact on mental health of university students. Psychological Studies, 66(3), 291-302. https://doi.org/10.1007/s12646-021-00610-1 Zolotareva, A. (2021). Psychometric examination of the Russian version of the Depression, Anxiety, and Stress scales-21. Psikhologicheskii Zhurnal, 42(5), 80-88. https://doi.org/10.31857/S020595920017077-0 (in Russian)
