CC BY
0РОССИЙСКИЙ ЖУРНАЛ ИНФОРМАЦИОННЫХ ТЕХНОЛОГИЙ В СПОРТЕ 2024 Т. 1 № 1 С. 21-42
DOI: https://doi.org/10.62105/2949-6349-2024-1-1-21-42
УДК 796.332
Принятие решений футбольными судьями без зрителей: психологические, коммуникативные и функциональные факторы Б. Обуди1, Б. Дольнего2, Е. Глебова3, М. Х. Абдоллахи4, Г. Гецци2
1 Университет Хорезми, Тегеран, Иран 2 Венгерский университет спортивных наук, Будапешт, Венгрия 3 Университет Париж Саклэ, Орсэ, Франция 4 Ширазский университет, Шираз, Иран
Аннотация
Целью этого исследования было определить приоритетность влияния COVID-19 и стадионов без зрителей на суждения футбольных судей, помощников футбольных судей, судей по мини-футболу и судей по пляжному футболу в Исламской Республике Иран. Для сбора данных использовались как качественный, так и количественный подходы. Качественный раздел включал интервью с 15 экспертами по судейству, в то время как количественный раздел включал футбольных судей мужского и женского пола, судей по мини-футболу и судей по пляжному футболу в Иране (n=339). После интервью разработанная исследователями анкета с 14 вопросами была распространена в электронном виде среди судей. Их спросили в целом о влиянии отсутствия зрителей на их выступление. Для выявления факторов и составляющих, влияющих на судей, был проведен факторный анализ с использованием программного обеспечения SPSS. Дополнительно был проведен подтверждающий факторный анализ первого и второго порядка (SEM) с использованием программного обеспечения LISREL для проверки компонентов. Наконец, выявленные факторы были ранжированы с использованием теста Фридмана. Было обнаружено, что стадионы без зрителей приносят пользу молодым и неопытным судьям, поскольку они могут более эффективно судить матчи. Формат матчей без зрителей также привел к меньшему психологическому давлению на судей, игроков и тренеров, поскольку не было зрителей, которые могли бы протестовать против ошибок или плохой игры.
Ключевые слова: за закрытыми дверями, принятие решений, COVID-19, судьи, зрители
RUSSIAN JOURNAL OF INFORMATION TECHNOLOGY IN SPORTS 2024 Vol. 1 No 1 P. 21-42
DOI: https://doi.org/10.62105/2949-6349-2024-1-1-21-42
Decision-making of football referees behind closed doors:
psychological, communicative, and functional factors B. Oboudi1, B. Dolnego2, E. Glebova3, M. H. Abdollahi4, G. Geczi2
1 Kharazmi University, Tehran, Iran 2 Hungarian University of Sport Sciences, Budapest, Hungary 3 University Paris Saclay, Orsay, France 4 Shiraz University, Shiraz, Iran
Abstract
This study aimed to prioritize the effects of COVID-19 and stadiums without spectators on the judgment of football referees, assistant football referees, futsal referees, and beach soccer referees in the Islamic Republic of Iran. Both qualitative and quantitative approaches were used to collect the data. The qualitative section involved interviews with 15 refereeing experts, while the quantitative section involved male and female football referees, futsal referees, and beach soccer referees in Iran (n=339). Following the interviews, a researcher-designed questionnaire with 14 questions was electronically distributed among the referees. They were asked in general about the impact of the absence of spectators on their performance. To identify the factors and components affecting referees, factor analysis was conducted using SPSS software. Additionally, first- and second-order confirmatory factor analysis (SEM) was performed using LISREL software to validate the components. Finally, the identified factors were ranked using the Friedman test. It was found that stadiums without spectators benefited young and inexperienced referees, as they were able to judge matches more effectively. The closed doors format also resulted in less psychological pressure on referees, players, and coaches, since there were no spectators to protest mistakes or poor performance.
Keywords: behind closed doors, decision-making, COVID-19, referees, spectators
Introduction
The physical presence of spectators, fans, and supporters in a stadium is considered crucial for all sporting events [1, 2]. The sports industry has been greatly affected by the COVID-19 pandemic [3, 4, 5], leading to the organization of events without spectators [1]. This has had a significant impact on the financial and marketing aspects [6] of professional football clubs, posing a serious threat [7]. The presence of spectators in stadiums has always played a pivotal
role in the success of teams, both in terms of providing an attractive and exciting competition for athletes and in generating income for clubs through sponsors, media attention, broadcasting rights, and overall quality of sports competitions [8, 9].
The impact of the absence of spectators in stadiums on referees' performance remains an ongoing subject of scientific investigation. Findings from various studies have yielded mixed results. Notably, it was concluded that spectators' presence in stadiums and the home advantage of teams primarily stemmed from the pressure applied by the audience on the match officials [10, 11]. However, these conclusions contradict the outcomes of a similar study carried out by [12]. Other studies have also demonstrated systematic bias favoring home teams in Brazilian, Italian, Spanish, and German elite football matches, particularly when spectators are present [13, 14, 15, 16, 17, 18, 19, 20, 21]. Referees' decisions tend to benefit these home teams, especially when they trail the visiting team or secure a victory. Referees appear inclined to extend match duration when the home team is behind, while hastening it when the home team comes to be victorious. Notably, fewer red and yellow cards are issued to home teams in such situations, all of which are influenced by the home advantage and the presence of spectators in the stadium. Consequently, research of this nature highlights the influence of spectators' presence and pressure on referees' decision-making, particularly in competitions where teams act as hosts. Thus, it would be imprudent to disregard the role of spectators in both the psychological and technical aspects of referees' performance during a match. Concurrently, sports spectators undeniably contribute significantly to the success of teams.
The presence of spectators in home games and their support for the home team have been recognized as factors that not only enhance player performance but also exert pressure on both the rival team and the referees [1, 5]. However, due to the ongoing coronavirus epidemic, stadiums have been devoid of spectators, necessitating an investigation into the impact of this absence on referee performance during match judgments. It is essential to acknowledge that referees, like other athletes, face numerous challenges amidst the COVID-19 pandemic, such as quarantine, isolation from loved ones, restricted interaction with fellow referees, psychological difficulties, and financial setbacks, including income reduction [4]. While players and coaches receive continuous support from their clubs and regular health check-ups to ensure their physical and mental well-being during this challenging time, referees, as integral sports figures, require distinctive attention and care across various aspects. Seeking guidance from experienced and knowledgeable psychologists is one recommended approach to enable referees to overcome any potential negative effects of the Corona epidemic on their performance and deliver optimal results during competitions [22, 23]. However, it is crucial to review the current situation and its implications resulting from COVID-19 from multiple perspectives, aiming to eliminate obstacles and resolve problems to improve the relationships between players, coaches, and referees, ultimately enhancing their performance in the most effective manner.
Upon review of the existing literature and research on football refereeing, it is apparent
that there is a lack of comprehensive studies, both domestically and internationally, that specifically explore the influence of the absence of football spectators on referees' performance and decision-making. The prevailing research in Iran primarily focuses on areas such as physical match performance, physical attributes, and decision-making during matches, as evidenced by the works of [24, 25, 26, 27]. Consequently, there is a notable absence of studies addressing this particular subject matter for Iranian referees, despite the conflicting evidence previously mentioned.
Examining various aspects of the effects of sports events without spectators on referees' performance can provide more detailed information to the field of sports management and sports psychology. Referees are crucial figures in sports competitions, making it important to investigate the factors that influence their different aspects. Referees in Iran, where video assistant referee systems are not yet implemented in football matches, have concerns about potential criticism for incorrect judgments. In the absence of necessary hardware and software facilities for referees, sports managers can negatively impact their performance and decision-making abilities. Therefore, it is necessary to study the effects of matches without spectators on referees' performance.
Since refereeing errors have always been a part of the game and referees face increased pressure in the presence of football spectators, leading to a potential rise in mistakes, the primary objective of this research is to investigate the impact of the absence of spectators in football stadiums on the performance of referees, especially according to the lack of research on this topic in Iran.
Theoretical background
Due to the COVID-19 pandemic, nearly all of the top football leagues worldwide suspended their competitions in mid-March 2020, and resuming these matches required playing without spectators present [5]. From a psychological standpoint, the absence of supporters on an unprecedented scale in the world's premier football leagues creates a distinctive opportunity to explore the influence of crowds on referees' decision-making, which is a novel situation in the history of professional football [28].
Home advantage and the attendance of spectators in stadiums are two crucial factors in football that significantly affect the performance of referees. Extensive support in the scientific literature exists for the hypothesis that the home advantage can be mostly accounted for by the social pressure exerted on referees, primarily through the vocal support of the home team's crowd during matches. The ongoing debate revolves around whether referees tend to exhibit bias in favor of the home team or against the away team [20, 28, 29, 30]. The home advantage effect can be plausibly explained from psychological and evolutionary standpoints, emphasizing the impact of social pressure and conformity. Research in social psychology demonstrates that individuals have
a tendency to conform to the majority's opinion because they (often unconsciously) perceive it to be more accurate than their own interpretation of ambiguous situations, resulting in conforming behavior [31]. Accordingly, studies have indicated that various factors such as crowd size [32], crowd density [33, 34], and stadium properties, specifically the distance to the field [35], exhibit a positive correlation with the extent of the home advantage and subsequently, their influence on referee decisions. In a conducted experimental study that explored the impact of crowd noise on referees' decision-making in football, it was discovered that referees, when exposed to the noise generated by the crowds, displayed heightened uncertainty in their judgments and ultimately awarded significantly fewer fouls (15.5%) against the home team, in contrast to referees who evaluated the same game in a soundless environment [11].
Other studies have indicated that the size of the crowd has a direct impact on the frequency of first yellow cards given to the away team in Cup final games [36]. Furthermore, referees have been observed to issue red cards and award penalties more frequently against the away team [37], emphasizing the positive influence of training on the objectivity and decision-making of referees in order to counterbalance the impact of social pressure from spectators [38]. Prior to the COVID-19 pandemic, which necessitated matches being played without spectators, research was conducted on games played under such conditions and their effects on refereeing. Specifically, a study focusing on football matches played behind closed doors due to safety concerns following hooligan incidents in Sicily (Italy) revealed that, under normal circumstances, home teams tend to receive more favorable decisions from officials during matches [39]. They explain that "the home team is punished less severely than the away team across all outcomes in games with spectators, while being punished more severely than the away team across all outcomes in matches without spectators." Moreover, the analyses have indicated that the estimated bias effect is statistically significant for all outcomes related to the number of fouls, the number of yellow cards, and the number of red cards. These data strongly suggest that referees alter their behavior in matches without spectators rather than the players. However, other studies suggest that the crowd can emotionally influence not only the referees but also the players, for example, home players feeling motivated and away players feeling pressured [32, 40, 41, 42].
Numerous studies have been conducted on the impact of the COVID-19 pandemic on football, with a particular focus on the home advantage effect. The majority of these studies suggest that the home advantage tends to diminish when matches are played without spectators present [20, 28, 30, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52]. In [43] it was claimed that «without a crowd, fewer cards were awarded to the away teams, reducing home advantage, and these results have implications for the influence of social pressure and crowds on the neutrality of decisions». Additionally, it has been found that ghost games, where matches are played in empty stadiums, may directly impact the non-verbal behavior of professional football players, staff, and officials [28]. Specifically, they observed 20 percent fewer emotional situations in ghost games compared to regular matches with fans present. This means that players, staff, and officials were less
engaged in behaviors such as verbal disputes and arguments with each other. However, a few studies suggest that the home advantage effect remains relatively unchanged during ghost games [41, 53, 54].
Based on previous and current studies investigating the absence of spectators in football, the authors were motivated to explore the significant effects of COVID-19-related football matches played behind closed doors in Iran on various dimensions of referees. This study aims to examine and discuss these effects from a psychological, communicative, and functional perspective. It is worth noting that football, futsal, and beach soccer hold immense popularity in Iran, with the leagues of these sports receiving substantial attention from audiences and media. However, the COVID-19 pandemic and the subsequent matches without spectators, coupled with the absence of VAR in Iran, have created a need for further investigation into the performance of referees.
Method
The objective of this study was to determine and prioritize the impacts of a behind-closed-doors format and stadiums without spectators on the decision-making of football referees, assistant football referees, futsal referees, and beach soccer referees in the Islamic Republic of Iran. This research aims to have practical implications and utilizes a descriptive-survey data collection method through sequential exploratory design research with a mixed approach. Both qualitative and quantitative methods were employed for data collection, with the qualitative segment involving refereeing experts as the population, and the quantitative segment consisting of male and female football, futsal, and beach soccer referees from the Islamic Republic of Iran.
Data collection
The qualitative data collection tools for this study included semi-structured interviews, while the quantitative part involved a researcher-made questionnaire. It is important to note that the qualitative sample consisted of 15 experts and referees specializing in football, futsal, and beach soccer. The selection of participants was based on targeted theoretical and snowball techniques. Furthermore, the interviews continued until theoretical saturation was reached. In the quantitative section, a total of 339 referees were chosen as the available samples. Following the identification of factors through interviews, the reliability of the interviews was assessed using the intra-subject agreement study method, resulting in a reliability score of 0.79. Subsequently, a researcher-made questionnaire was developed, consisting of 14 questions utilizing a 5-point Likert scale. This questionnaire was distributed electronically among male and female referees involved in football, futsal, and beach soccer. These referees had prior experience judging in various leagues (provincial, third division, second division, first division, premier league) and international competitions amidst the COVID-19 epidemic.
Data analysis
To analyze the data, the team employed descriptive and inferential statistical methods. In order to identify the factors and components influencing referee's judgment, they utilized exploratory factor analysis through SPSS version 22 software. To validate these components, first and second-order confirmatory factor analysis (SEM) was conducted using LISREL software version 8.8. The identified factors were then ranked using Friedman's ranking test.
Results
The descriptive analysis of the demographic information of the referees is presented in Table 1. According to the gathered data, out of the 339 participants in this research, 288 (85%) were men while 51 (15%) were women. Among them, 24 referees were under 25 years old, 98 fell within the age range of 25 to 30 years, 100 were between 30 and 35 years old, 71 were between 35 and 40 years old, and finally, 46 referees were over 40 years old. Regarding marital status, 228 referees were married and 111 were single. Moving on to educational qualifications, 11 referees held a doctorate, 107 possessed a master's degree, 172 had a bachelor's degree, 33 held a post-graduate diploma, and 16 referees possessed a diploma. In terms of questioning their main role in judging, the survey revealed that out of 339 respondents, 76 participants (23%) were identified as football referees, 35 (10%) as assistant football referees, 45 (13%) as beach soccer referees, and the majority, 183 participants (54%), as futsal referees. Regarding the highest category of judgment in the competition, the results showed that 14 referees (4%) had experience in refereeing international matches, 146 (43%) in the Premier League, 114 (34%) in the first division of the country, 22 (7%) in the second division, 27 (8%) in the third division, and 16 referees (4%) in the provincial league. When asked about their highest qualification and degree, the survey found that 16 referees (5%) held international referee degrees, 207 (61%) had national degrees, 89 (26%) held the first degree, 20 (6%) had a second degree, and only 7 referees (2%) possessed a third degree. In terms of satisfaction with the profession of refereeing, an overwhelming majority of the participants, specifically 336 individuals (more than 99%), expressed their fondness for the profession, while only 3 participants (less than 1%) indicated a lack of interest. When asked if refereeing was their main occupation, 319 (94%) stated that their primary job was not refereeing, while a smaller number of 20 participants (6%) confirmed that it was indeed their main occupation. Lastly, in relation to their satisfaction with the salary received for officiating competitions, a significant majority of 316 individuals (93%) were dissatisfied with their salary, while only 23 participants (7%) expressed satisfaction with their earnings.
In addition to analyzing the participants' demographic information, our exploratory factor analysis of the questionnaire revealed that the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was calculated to be 0.924, which is well above the recommended threshold of 0.5. This indicates that the sample size used in our analysis is suitable for conducting factor analysis.
Table 1 — Descriptive statistics of participants' answers to questions
Question Option Frequency Percent
Gender? Men 288 85
Women 51 15
Total 339 100
Marital status? Married 228 67
Single 111 33
Total 339 100
How old are you? Under 25 24 7
Between 25 and 30 98 29
Between 30 and 35 100 29
Between 35 and 40 71 21
Over 40 46 14
Total 339 100
Educational qualifications? Ph.D. 11 3
Master 107 31
Bachelor 172 51
Post-graduate diploma 33 10
Diploma 16 5
Total 339 100
What is your main role in judging? Football referees 76 23
Assistant football referees 35 10
Beach soccer referees 45 13
Futsal referees 183 54
Total 339 100
What was the highest category in your International matches 14 4
judging experience? Premier League in Iran 146 43
The first division of Iran 114 34
The second division of Iran 22 7
The third division of Iran 27 8
Provincial league 16 4
Total 339 100
What is your highest qualification and International referee degree 16 5
degree? National referee degree 207 61
The first degree 89 26
The second degree 20 6
The third degree 7 2
Total 339 100
Do you like the profession of refereeing? Yes 336 99
No 3 1
Total 339 100
Is your main job referring? Yes 20 6
No 319 94
Total 339 100
Are you satisfied with the salary received Yes 23 7
for judging the competitions? No 316 93
Total 339 100
It is worth mentioning that Bartlett's test, which assesses the correlation between research variables, also yielded significant results at this stage. This further supports the validity of our factor analysis. Upon conducting varimax rotation, we identified three factors in relation to the questionnaire's 14 items. According to the Kaiser criterion (eigenvalues greater than 1), these factors provide a logical representation of the research and account for a significant amount of variance (70.88%) as outlined in Table 2.
Table 2 — Extracted components final factor analysis of the questionnaire
Initial eigenvalues Rotation Sums
Factor Total % of Variance Cumulative % Total % of Variance Cumulative %
1 7.48 53.43 53.43 4.23 30.20 30.20
2 1.34 9.58 63.02 2.88 20.58 50.78
3 1.10 7.87 70.88 2.81 20.10 70.88
Figure 1. Second-order confirmatory factor analysis
Table 3 displays the factor loadings matrix for each question following Varimax rotation. According to this table, there are four questions (questions 1 to 4) that pertain to judges' psychological factors, three questions (questions 5 to 7) that pertain to their communication factors, and seven questions (questions 8 to 14) that pertain to their functional factors.
Table 3 - Factor load matrix of questionnaire on components extracted after rotation by Varimax method
Questions Factors Operat. Psychol. Commun.
1 Judging in stadiums without spectators has reduced the stress of referees during judging .84
2 The absence of spectators in the stadiums has reduced the fear and concern of the referees about a possible conflict with the spectators at the end of the match .83
3 The absence of spectators in the stadiums has put less psychological pressure on the referees .74
4 The absence of spectators in the stadiums has increased the confidence of the referees .54
5 The empty stadium made it easier for referees to communicate with players and coaches .81
6 The stadium empty of spectators has made more coordination between the referees and assistant referees during the match .83
7 The absence of spectators in the stadiums allows better communication between referees and better identification of obscene and aggressive players .83
8 The orientation and bias of the referees toward the host teams have decreased compared to the time before Corona .69
9 Matches without spectators have made the decisions of the referees to be balanced for the home and away teams .81
10 The empty stadium made it easy for the referees to control the game .73
11 Matches without spectators have made referees judge the match regardless of the teams' position in the table .74
12 The absence of spectators in the stadium has helped young and inexperienced referees in judging the matches .68
13 The absence of spectators in the stadiums has made it easier for referees to fine players and coaches .70
14 The absence of spectators in the stadiums has led to an improvement in referees' performance .70
It is important to note that the standard model was used to examine the relationships between the constructs and the questionnaire's questions. Initially, first-order factor analysis was conducted to explore the relationships between the constructs. If the factor loadings for each construct exceeded 0.3, it indicated that the question had a strong explanatory power. Table 4 illustrates that all 14 questions in the questionnaire have T-values greater than 1.96
and factor loadings greater than 0.3. Therefore, the findings of this section confirm the accurate interpretation of the question dimensions, thereby validating all the questionnaire questions.
Table 4 — Investigating the factor loads of the questions Method
Questions Factors Factor loads T-value Result
1 Judging in stadiums without spectators has reduced the stress of referees during judging .74 15.08 confirmation
2 The absence of spectators in the stadiums has reduced the fear and concern of the referees about a possible conflict with the spectators at the end of the match .78 16.16 confirmation
3 The absence of spectators in the stadiums has put less psychological pressure on the referees .82 17.58 confirmation
4 The absence of spectators in the stadiums has increased the confidence of the referees .78 16.16 confirmation
5 The empty stadium made it easier for referees to communicate with players and coaches .75 15.55 confirmation
6 The stadium empty of spectators has made more coordination between the referees and assistant referees during the match .86 18.63 confirmation
7 The absence of spectators in the stadiums allows better communication between referees and better identification of obscene and aggressive players .87 19.05 confirmation
8 The orientation and bias of the referees toward the host teams have decreased compared to the time before Corona .68 13.87 confirmation
9 Matches without spectators have made the decisions of the referees to be balanced for the home and away teams .84 18.54 confirmation
10 The empty stadium made it easy for the referees to control the game .86 19.29 confirmation
11 Matches without spectators have made referees judge the match regardless of the teams' position in the table .72 14.87 confirmation
12 The absence of spectators in the stadium has helped young and inexperienced referees in judging the matches .62 12.18 confirmation
13 The absence of spectators in the stadiums has made it easier for referees to fine players and coaches .78 16.60 confirmation
14 The absence of spectators in the stadiums has led to an improvement in referees' performance .78 16.73 confirmation
To further validate the model and determine the factors and their impact on the referees' judgment, second-order confirmatory factor analysis was employed. The results of this analysis
are presented in Figure 1. All three psychological, communicative, and functional factors have an influence on referees' judgment during the coronavirus epidemic (Figure 1). Furthermore, for detailed information, the T-value and the path coefficient of these factors can be found in Table 5.
Table 5 — The T-value and the path coefficient of factors
Factors Psychological Communicational Operational
T-value 13.04 11.51 11.98
Path coefficient .91 .75 .88
Result Confirmation Confirmation Confirmation
Table 6 - Model fit indices
RMSEA GFI IFI CFI NFI NNFI Chi-square/df
.068 Favorable .96 Favorable .99 Favorable .99 Favorable .98 Favorable .98 Favorable 2.55 Favorable
Eventually, fit indices were employed to assess the factor structure of the proposed model. The results presented in Table 6, reveal that the Normed Chi-square, Root Mean Square Residual (RMR), Goodness of fit index (GFI), Normed Fit Index (NFI), Non-Normed Fit Index (NNFI), Incremental Fit Index (IFI), Comparative Fit Index (CFI), and Root Mean Square Error of Approximation (RMSEA) fall within the optimal range. These values indicate a favorable fit of the research model, thereby confirming the acceptability of the chosen factor structure. Furthermore, the data obtained from the research lends support to the underlying theoretical model.
After conducting exploratory and confirmatory factor analysis, the subsequent phase involved ranking the identified factors utilizing Friedman's test (p=.000, df=13, chi-square=770.0; Table 7).
Discussion
The aim of this research was to ascertain and prioritize the impacts of COVID-19 and the absence of spectators in stadiums on the decision-making of football referees, assistant football referees, futsal referees, and beach soccer referees in the Islamic Republic of Iran. The principal objective of this study was to address the question of how the coronavirus pandemic and the lack of spectators in stadiums have affected the communication, psychological, and performance aspects of referees, as well as determining the ranking of these factors.
Table 7 — Ranking of the identified factors on the judgment of referees during the COVID-19 pandemic
Factors Rank mean
The absence of spectators in the stadium has helped young and inexperienced 10.21
referees in judging the matches
The absence of spectators in the stadiums allows better communication 9.43
between referees and better identification of obscene and aggressive players
The stadium empty of spectators has made more coordination between the 8.58
referees and assistant referees during the match
The absence of spectators in the stadiums has reduced the fear and concern of 8.42
the referees about a possible conflict with the spectators at the end of the
match
Judging in stadiums without spectators has reduced the stress on referees 7.97
during judging
The empty stadium made it easy for the referees to control the match 7.86
The empty stadium made it easier for referees to communicate with players 7.58
and coaches
The absence of spectators in the stadiums has increased the confidence of the 7.36
referees
The absence of spectators in the stadiums has put less psychological pressure 7.18
on the referees
The absence of spectators in the stadiums has made it easier for referees to fine 7.11
players and coaches
The absence of spectators in the stadiums has led to an improvement in 7.07
referees' performance
Matches without spectators have made the decisions of the referees to be 6.15
balanced for the home and away teams
Matches without spectators have made referees judge the match regardless of 5.15
the teams' position in the table
The orientation and bias of the referees toward the host teams have decreased 4.92
compared to the time before Corona
The analysis of the findings revealed that referees' judgment during the coronavirus pandemic and matches without spectators was influenced by 14 main factors, which were categorized into three key components: communication, psychological, and functional. All three components were shown to impact the referees' judgment. However, in terms of significance, psychological factors had the highest impact with a value of 0.91, followed by functional factors with a value of 0.88, and lastly communication factors with a value of 0.75. In the ranking section analyzing the psychological, functional, and communication aspects, it was discovered that the absence of spectators in stadiums has significantly aided young and inexperienced referees in their match judgments. This factor claimed the top spot with an impressive average score of 10.21. The intense pressure from spectators and the charged atmosphere prevalent in stadiums, particularly
during important matches, inflict significant anxiety and stress upon referees, making their performance challenging. This issue applies to referees across various sports disciplines, where protests and even physical assaults against referees occur despite their competent performances. Consequently, sports events and high-stakes competitions create arduous conditions for aspiring referees. However, the ongoing coronavirus epidemic has presented a unique opportunity for young and inexperienced referees to officiate matches without the burden of audience pressure. The findings from this segment align with the research conducted by [22, 23].
The absence of spectators in the stadiums proved to be of significant importance, ranking second with an average score of 9.43. This phenomenon allowed referees to better identify players engaging in obscene and aggressive behavior. Without the noise and distractions caused by spectators, the voices of players and coaches became more audible to both the referees and officials overseeing the sporting events, as well as television viewers. This circumstance greatly facilitated the referees' ability to identify and handle misconduct on the field.
Additionally, the empty stadiums contributed to enhanced coordination between referees and assistant referees during the matches. This factor, ranking third with an average score of 8.58, complemented the aforementioned second factor. Effective coordination among referees is crucial for their optimal performance and the overall quality of the competition. Historically, numerous challenges hindered referees' performance, including limited communication capabilities and outdated refereeing equipment and facilities. However, the absence of spectators in the stadiums has mitigated some of these issues by allowing referees to maintain better focus and concentration.
Overall, the absence of spectators in stadiums has offered numerous benefits for referees, including improved identification of misconduct and enhanced coordination among officiating officials. This circumstance has partially compensated for existing hardware deficiencies and ultimately contributed to a more efficient and fair sporting environment.
Furthermore, the absence of spectators in the stadiums has alleviated the fear and anxiety among referees regarding potential conflicts with spectators at the conclusion of matches. This aspect holds significant importance for referees, with an average ranking of 4 out of 42.8, as supported by [23] research findings. This factor highlights a recurring issue that unfortunately tends to occur predominantly in Iranian football and futsal competitions. Referees lack the security and tranquility they need in the presence of spectators, particularly in lower leagues and categories. There is a constant concern of being subjected to attacks from fans, players, and technical staff after matches. Moreover, the introduction of the virtual space has further amplified the pressures faced by referees, intensifying their sense of apprehension. Alongside coaches and athletes, referees constitute an essential faction in sporting events, shouldering the most arduous responsibilities. Refereeing, across all sports, has always been inherently associated with anxiety and stress.
The absence of spectators in stadiums during judging has resulted in a decrease in stress levels experienced by referees. This factor was ranked fifth in terms of importance, with an average rating of 7.97. Similar findings were observed in studies conducted by [17, 22, 23].
Referees often face psychological pressure due to external factors such as players, technical staff, and especially spectators, which can negatively impact their performance. However, the absence of spectators has weakened this psychological factor for referees. The judges' primary concern revolves around psychological issues such as fear and anxiety of making mistakes during the match. To address this, it is important to enhance security measures during competitions, implement programs to address psychological issues, ensure adequate remuneration for judges, improve their quality of life, and most importantly, provide conditions for video referees in Iran. By considering these factors, the psychological pressure on referees can be effectively reduced.
However, one noteworthy aspect among the identified factors is the impact of matches held without spectators on referee decisions, regardless of the teams' standings. On average, referees ranked these matches at 5.15, placing them 13th in importance. Additionally, there is a diminishing trend in referee bias towards host teams compared to pre-COVID times, with an average rating of 4.92, placing them fourth. These findings contradict the research results of [14, 15, 17, 18, 19, 20]. Iranian referees argue that there is no difference in judgment, prejudice, or bias towards teams, whether the matches are held with or without spectators in the stadiums, or irrespective of the teams' positions in the table.
Based on the identified factors, it can be argued that the current circumstances of spectator absence in stadiums have resulted in reduced psychological pressure on referees, players, and coaches. Essentially, the absence of spectators means that referees, who may unintentionally make glaring errors or perform poorly during matches, are shielded from the pressure of protests from the audience. This research has revealed that the lack of spectators in stadiums serves as a catalyst for increased motivation and confidence among young and less experienced referees when officiating sensitive and significant matches. Although there is a possibility of mistakes by these referees in Premier League matches, this initiative can effectively provide valuable support to the field of refereeing.
In addition to the aforementioned cases, the role of hardware and software facilities and equipment should not be overlooked when aiming to enhance referees' performance and decision-making abilities. Communication systems, goal-line technology, and, most notably, the implementation of a Video Assistant Referee (VAR) system can significantly enhance referees' performance and alleviate psychological pressure, as well as diminish protests from spectators, coaches, and players. The absence of a video assistant referee in the Persian Gulf Premier League, in comparison to professional leagues worldwide, has resulted in refereeing mistakes becoming a prominent cause for protests against referees and a lack of success for teams. Hence, it is imperative for the country's sports officials and managers to not only provide modern facilities and equipment for referees but also prioritize their financial, educational, and, most importantly, psychological needs.
The presence of spectators has been a critical factor contributing to the success of sports teams when playing at home. Thus, the absence of spectators significantly diminishes the home advantage. Additionally, based on previous research, it can be argued that in matches played
without spectators, commonly referred to as behind-closed-doors matches, referees experience less social pressure from the home crowd.
The findings of this study offer a scientific foundation for improving the structure and necessary conditions to optimize the judgment abilities of Iranian football referees, thereby reducing the influence of spectators on referees' decision-making in the future.
Limitations
As the research data and responses were obtained through the use of questionnaires, it was not feasible to obtain comprehensive and genuine insights into the referees' perspectives. Their responses may be biased and tend to align with specific directions in addressing certain questions. In contrast, the utilization of neuroscience methodologies can provide a more profound understanding of referees' reactions and responses to inquiries, surpassing the limitations of self-reports and questionnaires.
As this study was primarily aimed at exploration, it was necessary to undertake certain procedures to ascertain the influential factors. Conducting confirmatory factor analysis (CFA) on the same dataset used for the initial exploratory factor analysis (EFA) is a common practice. However, it may not always be advisable to follow this approach. In cases where there is insufficient data to partition into subsets for EFA and CFA, employing cross-validation may be a viable option to ensure the broader applicability of the results.
Another key issue to consider when addressing the question of the decision-making of football referees behind closed doors. Various approaches and constructs are evident in the academic literature on this topic [28, 54]. Some studies utilize the win ratio or points gained/lost to assess the impact on referee performance, whereas others analyze the distribution of yellow and red cards to home and away teams. Given that certain Iranian football teams have limited spectators in their stadiums and perceive no difference between matches with or without spectators, referees may be influenced by popular teams, as most spectators in Iran are fans of a few specific teams (Esteghlal, Persepolis, Sepahan, Tractor). However, the present study does not attempt to pass judgment on these various approaches. Instead, it contends that these circumstances should not affect referees' performance and emphasizes the importance of fairness in all situations.
The Iranian national football team ranks second in Asia and is positioned within the top 25 teams globally according to the FIFA World Ranking. Additionally, the Iranian Football League, also known as the Persian Gulf League, is recognized as the fourth most influential football league in Asia. However, it currently lacks the necessary technological infrastructure for football referees, which consequently impacts their performance. It is important to note that the findings of this research, conducted in Iran, may not be universally applicable to other countries. Nevertheless, similar studies can be conducted in other nations to compare results and factors influencing the performance of football referees.
Conclusion
The COVID-19 pandemic has been one of the most significant crises since the last world war, impacting all aspects of society, including sports and professional football. This has raised numerous practical and scientific questions, including the economic viability of clubs without spectator revenues and the noticeable changes in stadium atmosphere. Early reports suggested that player behavior, particularly towards referees, was affected, and recent studies have confirmed that the absence of crowds does indeed influence the nonverbal behavior of players, staff, and referees.
Almost all football leagues worldwide have played games without fans since at least mid-2020. A study was conducted to examine the impact of football spectators on the performance of Iranian football referees. In addition to summarizing the key findings of previous research in this area, the study provided a thorough analysis of the important factors affecting the function of Iranian referees.
The findings of this study indicate that matches played without spectators during the COVID-19 pandemic, across various levels of football, futsal, and beach soccer leagues, significantly differ from regular matches that have spectators. Historically, the interactions between coaches, players, and referees have been known to be contentious. However, due to the absence of crowd noise and the presence of pitch side microphones that can potentially capture any sound or conversation, including abusive language directed at referees, players and coaches may become acutely aware of the consequences. Consequently, their behavior may adapt in this more controlled environment. These changes have the potential to bring about positive impacts, not only in high-profile leagues and professional football, but also in lower-level football leagues. By reducing negative behaviors in elite leagues worldwide, there may be a potential shift in the dynamics between referees, players, and coaches at lower levels of the game as well.
Verbal and physical abuse significantly contribute to the discontinuation of referees, affecting the ability of national football associations to recruit and retain referees in lower-level leagues. Efforts to reduce such abuse would greatly support their mission. However, there is a potential rise in other forms of abuse, such as online abuse through social media networks, which remains unexplored in academic research. Referees are often seen as an out-group in football, having distinct goals and responsibilities compared to players, coaches, and spectators. While referees prioritize enforcing the laws of the game, they have little vested interest in the outcome of any specific team. In contrast, players, coaches, and spectators are emotionally invested in their respective teams' success. Unlike players, not all referees receive a monthly salary or have contractual agreements. These divergent interests often lead to tension and subsequent abuse directed towards referees. The concern lies in determining whether the verbal or physical abuse experienced by a referee would ultimately drive them away from the game altogether.
References
1. Tamir I. There's no sport without spectators - viewing football games without spectators during the COVID-19 pandemic. Frontiers in Psychology, 2022, 13, Article 860747. DOI: https://doi.org/10.3389/fpsyg.2022.860747
2. Glebova E., Zare F., Desbordes M., Geczi G. 'If there is no football, then we have nothing to discuss': a phenomenological study on football fandom and COVID-19 pandemic. International Journal of Sport Management and Marketing, 2023, 23 (5), pp. 373-390. DOI: https://doi.org/10.1504/IJSMM.2023.133158
3. Parnell D., Widdop P., Bond A., Wilson R. COVID-19, networks and sport. Managing Sport and Leisure, 2020, 27(1-2), pp. 78-84. DOI: https://doi.org/10.1080/23750472.2020.1750100
4. Webb T., Dicks M., Thelwell R., van der Kamp J., Rix-Lievre G. An analysis of soccer referee experiences in France and the Netherlands: abuse, conflict, and level of support. Sport Management Review, 2021, 23(1), pp. 52-65. DOI: https://doi.org/10.1016/j.smr.2019.03.003
5. Glebova E., Zare F., Desbordes M., Geczi G. COVID-19 sport transformation: new challenges and new opportunities. Physical Culture and Sport. Studies and Research, 2022, 95 (1), pp. 54-67. DOI: https://doi.org/10.2478/pcssr-2022-0011
6. Glebova E., Desbordes M. Identifying the role of digital technologies in sport spectators customer experiences through qualitative approach. Athens Journal of Sports, 2021, 8 (2), pp. 141-160. DOI: https://doi.org/10.30958/ajspo.8-2-3
7. Reade J.J., Singleton C. Demand for public events in the COVID-19 pandemic: a case study of European football. European Sport Management Quarterly, 2020, 21 (3), pp. 391-405. DOI: https://doi.org/10.1080/16184742.2020.1841261
8. Dantas F., Borges A., Silva R. Impact of UEFA Champions League and UEFA Europa League on financial sustainability - case study of two small football Portuguese teams. Sustainability, 2020, 12 (21), Article 9213. DOI: https://doi.org/10.3390/su12219213
9. Herold E., Boronczyk F., Breuer C. Professional clubs as platforms in multi-sided markets in times of COVID-19: the role of spectators and atmosphere in live football. Sustainability, 2021, 13 (4), Article 2312. DOI: https://doi.org/10.3390/su13042312
10. Boyko R.H., Boyko A.R., Boyko M.G. Referee bias contributes to home advantage in English Premiership football. Journal of Sports Sciences, 2007, 25(11), pp. 1185-1194. DOI: https://doi.org/10.1080/02640410601038576
11. Nevill A.M., Balmer N.J., Williams A.M. The influence of crowd noise and experience upon refereeing decisions in football. Psychology of Sport and Exercise, 2002, 3(4), pp. 261-272. DOI: https://doi.org/10.1016/S1469-0292(01)00033-4
12. Johnston R. On referee bias, crowd size, and home advantage in the English soccer Premiership. Journal of Sports Sciences, 2008, 26 (6), pp. 563-568. DOI: https://doi.org/10.1080/02640410701736780
13. Dawson P., Dobson S., Goddard J., Wilson J. Are football referees really biased and inconsistent?: evidence on the incidence of disciplinary sanction in the English Premier League. Journal of
the Royal Statistical Society Series A: Statistics in Society, 2007, 170(1), pp. 231-250. DOI: https://doi.Org/10.1111/j.1467-985X.2006.00451.x
14. Dohmen T. J. Do professionals choke under pressure? Journal of Economic Behavior & Organization, 2008, 65 (3-4), pp. 636-653. DOI: https://doi.org/10.1016/j.jebo.2005.12.004
15. Dohmen T., Sauermann J. Referee bias. Journal of Economic Surveys, 2016, 30 (4), pp. 679-695. DOI: https://doi.org/10.1111/joes.12106
16. Garicano L., Palacios-Huerta I., Prendergast C. Favoritism under social pressure. The Review of Economics and Statistics, 2005, 87(2), pp. 208-216. DOI: https://doi.org/10.1162/0034653053970267
17. Liu T., Garcia-De-Alcaraz A., Zhang L., Zhang Y. Exploring home advantage and quality of opposition interactions in the Chinese Football Super League. International Journal of Performance Analysis in Sports, 2019, 19(3), pp. 289-301. DOI: https://doi.org/10.1080/24748668.2019.1600907
18. Rocha B., Sanches F., Souza I., Carlos Domingos da Silva J. Does monitoring affect corruption? Career concerns and home bias in football refereeing. Applied Economics Letters, 2013, 20(8), pp. 728-731. DOI: https://doi.org/10.1080/13504851.2012.736938
19. Scoppa V. Are subjective evaluations biased by social factors or connections? An econometric analysis of soccer referee decisions. Empirical Economics, 2007, 35(1), pp. 123-140. DOI: https://doi.org/10.1007/s00181-007-0146-1
20. Sors F., Grassi M., Agostini T., Murgia M. The sound of silence in association football: Home advantage and referee bias decrease in matches played without spectators. European Journal of Sport Science, 2021, 21 (12), pp. 1597-1605. DOI: https://doi.org/10.1080/17461391.2020.1845814
21. Sutter M., Kocher M. G. Favoritism of agents - The case of referees' home bias. Journal of Economic Psychology, 2004, 25 (4), pp. 461-469. DOI: https://doi.org/10.1016/S0167-4870(03)00013-8
22. Webb T. The future of officiating: analysing the impact of COVID-19 on referees in world football. Soccer & Society, 2020, 22 (1-2), pp. 12-18. DOI: https://doi.org/10.1080/14660970.2020.1768634
23. Webb T., Gorczynski P. Factors influencing the mental health of sports match officials: The potential impact of abuse and a destabilised support system from a global context. In: Routledge Handbook of Athlete Welfare. Routledge, 2020, pp. 289-299.
24. Oboudi A., Ghahfarokhi E.A. The effects of 90 sport program on Iranian football referees. Sport Management Journal, 2021, 13(2), pp. 341-356. DOI: https://doi.org/10.22059/jsm.2021.117414.1307
25. Bonyadifard M., Ghahfarokhi E.A., Dehghan A., Jalali M. Developing a strategic plan for the referees committee of the iranian football federation. Strategic Studies on Youth and Sports, 2022. DOI: https://doi.org/10.22034/ssys.2022.1102.1754
26. Salehi H., Mohseni M. Anxiety level among Iranian football referees: The role of self-consciousness and coping styles. Research in Sport Management and Motor Behavior, 2017, 7(14), pp. 16-25 (in Persian). URL: http://jrsm.khu.ac.ir/article-1-2655-en.html
27. Abdi H., Farahani M.J., Sajjadi S.N. A comparative study of Status Football's Referee in Iran and England. Sports Management Journal, 2008, 2 (4), pp. 119-135 (in Persian).
28. Leitner M.C., Richlan F. No fans-no pressure: referees in professional football during the COVID-19 pandemic. Frontiers in Sports and Active Living, 2021, 3, Article 720488. DOI: https://doi.org/10.3389/fspor.2021.720488
29. Pollard R. Home advantage in football: a current review of an unsolved puzzle. The Open Sports Sciences Journal, 2008, 1 (1), pp. 12-14. DOI: https://doi.org/10.2174/1875399X00801010012
30. Wunderlich F., Weigelt M., Rein R., Memmert D. How does spectator presence affect football? Home advantage remains in European top-class football matches played without spectators during the COVID-19 pandemic. PLoS One, 2021, 16(3), Article e0248590. DOI: https://doi.org/10.1371/journal.pone.0248590
31. Cialdini R.B., Goldstein N.J. Social influence: compliance and conformity. Annual Review of Psychology, 2004, 55, pp. 591-621. DOI: https://doi.org/10.1146/annurev.psych.55.090902.142015
32. Nevill A.M., Newell S.M., Gale S. Factors associated with home advantage in English and Scottish soccer matches. Journal of Sports Sciences, 1996, 14(2), pp. 181-186. DOI: https://doi.org/10.1080/02640419608727700
33. Agnew G.A., Carron A.V. Crowd effects and the home advantage. International Journal of Sport Psychology, 1994, 25, pp. 53-62 (in Persian).
34. Goumas C. Home advantage and referee bias in European football. European Journal of Sport Science, 2014, 14S, pp. 243-249. DOI: https://doi.org/10.1080/17461391.2012.686062
35. Unkelbach C., Memmert D. Crowd noise as a cue in referee decisions contributes to the home advantage. Journal of Sport and Exercise Psychology, 2010, 32(4), pp. 483-498. DOI: https://doi.org/10.1123/jsep.32.4.483
36. Downward P., Jones M. Effects of crowd size on referee decisions: Analysis of the FA Cup. Journal of Sports Sciences, 2007, 25 (14), pp. 1541-1545. DOI: https://doi.org/10.1080/02640410701275193
37. Pollard R., Armatas V. Factors affecting home advantage in football World Cup qualification. International Journal of Performance Analysis in Sport, 2017, 17(1-2), pp. 121-135. DOI: https://doi.org/10.1080/24748668.2017.1304031
38. Nevill A., Webb T., Watts A. Improved training of football referees and the decline in home advantage post-WW2. Psychology of Sport and Exercise, 2013, 14(2), pp. 220-227. DOI: https://doi.org/10.1016/j.psychsport.2012.11.001
39. Pettersson-Lidbom P., Priks M. Behavior under social pressure: Empty Italian stadiums and referee bias. Economics Letters, 2010, 108 (2), pp. 212-214. DOI: https://doi.org/10.1016/j.econlet.2010.04.023
40. George A. The perception of home advantage and its effect on selfconfidence from the viewpoint of female football players. Unpublished undergraduate dissertation. Cardiff: Cardiff Metropolitan University, 2015.
41. Almeida L.G., Oliveira M. L., Silva C.D. Uma analise da vantagem de jogar em casa nas duas principais divisoes do futebol profissional brasileiro. Revista Brasileira de Educaçao Fisica e Esporte, 2011, 25 (1), pp. 49-54 (in Portuguese).
42. Leite W., Pollard R. Comparison of home advantage between level 1 and level 2 in women's football leagues. Journal of Anthropology of Sport and Physical Education, 2020, 4(4), pp. 9-13. URL: http://www.jaspe.ac.me/?sekcija=article&artid=162
43. Bryson A., Dolton P., Reade J., Schreyer D., Singleton C. Experimental effects of an absent crowd on performances and refereeing decisions during COVID-19. IZA Discussion Papers, 2020, Article 13578. DOI: http://dx.doi.org/10.2139/ssrn.3669507
44. Dilger A., Vischer L. No home bias in ghost games. Diskussionspapier des Instituts fur Organisationsökonomik, 2020, 7. DOI: http://dx.doi.org/10.2139/ssrn.3674876
45. Fischer K., Haucap J. Does crowd support drive the home advantage in professional football? Evidence from German ghost games during the COVID-19 pandemic. Journal of Sports Economics, 2021, 22 (8), pp. 982-1008. DOI: https://doi.org/10.1177/15270025211026552
46. Follert F., Daumann F., Passon L. Zur Bedeutung der Fans im professionellen Fußball im Lichte der COVID-19-Pandemie. Sciamus - Sport und Management, 2020, 2, pp. 28-42 (in German).
47. Hill Y., Van Yperen N.W. Losing the home field advantage when playing behind closed doors during COVID-19: change or chance? Frontiers in Psychology, 2021, 12, Article 658452. DOI: https://doi.org/10.3389/fpsyg.2021.658452
48. Sanchez A.J., Lavin J. M. Home advantage in European soccer without crowd. Soccer & Society, 2021, 22 (1-2), pp. 152-165. DOI: https://doi.org/10.1080/14660970.2020.1830067
49. Konaka E. Home advantage of European major football leagues under COVID-19 pandemic. arXiv preprint, 2021, arXiv:2101.00457. DOI: https://doi.org/10.48550/arXiv.2101.00457
50. McCarrick D., Bilalic M., Neave N., Wolfson S. Home advantage during the COVID-19 pandemic: Analyses of European football leagues. Psychology of Sport and Exercise, 2021, 56, Article 102013. DOI: https://doi.org/10.1016/j.psychsport.2021.102013
51. Santana H.A.P., Bettega O.B., Dellagrana R. An analysis of Bundesliga matches before and after social distancing by COVID-19. Science and Medicine in Football, 2021, 5S, pp. 17-21. DOI: https://doi.org/10.1080/24733938.2021.1903540
52. Scoppa V. Social pressure in the stadiums: Do agents change behavior without crowd support? Journal of Economic Psychology, 2021, 82, Article 102344. DOI: https://doi.org/10.1016/j.joep.2020.102344
53. Benz L.S., Lopez M.J. Estimating the change in soccer's home advantage during the Covid-19 pandemic using bivariate Poisson regression. AStA Advances in Statistical Analysis, 2023, 107(1-2), pp. 205-232. DOI: https://doi.org/10.1007/s10182-021-00413-9
54. Matos R., Monteiro D., Antunes R., Mendes D., Botas J., Clemente J., Amaro N. Home-advantage during COVID-19: an analysis in Portuguese football league. International Journal of Environmental Research and Public Health, 2021, 18(7), Article 3761. DOI: https://doi.org/10.3390/ijerph18073761
Аbout the authors
Behnam Oboudi - Doctor of Sport Management (sport marketing and media), Faculty of Physical Education and Sport Sciences, Kharazmi University,Tehran, Iran. E-mail: a.oboudi72@yahoo.com
Bâlint Dolnegö - Hungarian University of Sport Sciences, Budapest, Hungary. E-mail: dolnego.balint@tf.hu
Ekaterina Glebova - Doctor of Sport Management, Researcher, University Paris Saclay, Orsay, France.
E-mail: katrina.glebova@universite-paris-saclay.fr
Mohammad Hasan Abdollahi - Doctor of Medicine, Department of Physical Education & Sport Sciences, Shiraz University, Shiraz, Iran. E-mail: abdollahi.1975@yahoo.com
Gabor Geczi - Doctor of Philosophy, Hungarian University of Sport Sciences, Budapest, Hungary.
E-mail: gabor@tf.hu
Для цитирования:
Обуди Б., Дольнего Б., Глебова Е., Абдоллахи М.Х., Гецци Г. Принятие решений футбольными судьями без зрителей: психологические, коммуникативные и функциональные факторы // Российский журнал информационных технологий в спорте. - 2024. - T. 1, № 1. - С. 21-42 (на англ. яз.). DOI: https://doi.org/10.62105/2949-6349-2024-1-1-21-42 EDN: GLHRVC
Cite as:
Oboudi B., Dolnego B., Glebova E., Abdollahi M.H., Géczi G. Decision-making of football referees behind closed doors: psychological, communicative, and functional factors. Russian Journal of Information Technology in Sports, 2024, 1 (1), pp. 21-42. DOI: https://doi.org/10.62105/2949-6349-2024-1-1-21-42 EDN: GLHRVC
Received: 25.11.2023 Accepted: 14.02.2024 Published: 13.03.2024
