CC BY
53DOI: 10.46991/JOPS/2022.1.1.040
Political disinformation and hate speech on facebook: the
attitude of young Armenians towards modern cyber challenges
Viktor ya Melkonyan
Abstract
This research explores political disinformation and hate speech on Facebook with particular stress on the attitude of the Armenian young population after Velvet revolution in Armenia. The empirical results find that cyber disinformation and hate speech in Armenia are mainly provoked and encouraged by two general components; political and social. In the case of political factors, the ongoing domestic political situation and foreign influence interests are highlighted. Whereas, in the case of social factors, the level of media literacy and social media's raising role in society is emphasized.
The study also reveals that despite the unclear extent of social media's influence on people's political participation, engagement in different social media platforms has a significantly growing role among young people, particularly in terms of developing political knowledge, getting aware of daily political news, of following politicians' and political institutions' activities, of discussing ongoing developments and transformations, highlighting questions and sharing opinions.
Keywords: Cyber hate speech, political disinformation, political participation, political attitude, youth, Facebook, Armenia.
Introduction:
Social networking sites (SNSs) have benefited humanity with endless opportunities starting from access to information and communication till modern political activeness and participation. In parallel, the mass usage of the SNSs led to the growth of cyber negative manifestations, so-called 'social dark sides'. People encounter disinformation and hate speech on a daily basis. This tendency has influenced people's political behavior and provoked their political actions, yet additional data and research are required to analyze the specific effects and the extent of social media's influence on people's political knowledge and participation. However, something is indisputable; the immense growing role of social media in people's lives clearly demonstrates that any act of online communication and engagement can alter individual opinion by virtual norms.
* Viktorya Melkonyan is a PhD candidate of the Chair of Political Science of the Faculty of International Relations at Yerevan State University Email: melkonyanviktorya@gmail.com
Journal of Political Science: Bulletin of Yere\*an University, Vol. 1, No. 1, May 2022, Pp. 40-68 Received: 25.11.2021
Revised: 10.12.2021 Accepted: 26.12.2021 This work is licensed under a Creative Commons Attribution-NonCommercial 4.0
International Li«**. © The Author(s) 2022
There are lots of sources within international literature stressing the role of SNSs in the process of socialization and political education of people. The aim of this research is to demonstrate the peculiarities of political disinformation and hate speech in a newly developing virtual political landscape in Armenia as well as to highlight the importance of the comprehensive discussion of novel virtual challenges as a new field of study both from the perspective of journalism and political science. What is more significant, it serves as a profound study which discusses the interconnection between disinformation and hate speech.
The research consists of two parts. The first part is dedicated to the discussion of the outbreak of cyber political disinformation. The main focus of the second part is the revelation of political hate speech on Armenian Facebook landscape starting from the Velvet revolution in 2018.
The object of the research is Facebook as it comes first as the most popular SNS in Armenia. In turn, the primary focus on post-revolutionary period in Armenia is conditioned by exclusive social-political transformations. The young people (officially 18-35 age range) is the target of the research as they are the heaviest user of SNSs, hence the influence (both negative and positive) on them is the highest.
This research stands out by an exploration of the online environment of political disinformation and hate speech in Armenia thus providing comprehensive theoretical and empirical background on these modern virtual challenges from scientific perspectives. In parallel, the relationship between disinformation and hate speech is revealed for the examination of possible outcomes and patterns of the interaction.
The theoretical background of the research is enriched and completed with the practical case discussion. The particular emphasis is on the practical research of Armenian young people's attitude towards the above-mentioned negative phenomena. In addition, the tendency of Armenian young people's engagement in SNSs' main discourses and their perception about political participation and democracy is examined. For this reason, two methods of qualitative data; such as focus groups and online surveys are used.
Disinformation: Theoretical and Practical Implications: "I want to see how the phenomenon of turning to the people is strengthening in our culture. There are different methods to do that; referendum or plebiscite and they should never be considered as extremal methods. In 21st century it is priceless. First of all because we live in two different realities; virtual and material-real life. Virtual life seems to become our main life. Discussions, speeches, and a huge amount of fakes and you are not able to understand and differentiate the truth from the artificially evolved information. It is important for each power to take steps in order to make people to give up living on Facebook and Instagram, to come back to the Earth, communicate and share with opinions in a real life"1.
1 The Office to the President of the RA. 2020. "Interviews and press conferences." Accessed December 10, 2021. https://www.president.am/hy/interviews-and-press-conferences/item/2020/02/07/President-Armen-Sarkissians-interview-/.
The rise of fake and false information
"Post-truth" and "fake news" are terms that have been widely used since the end of the 19th century and even have been proclaimed as the words of the year in 2016 and 2017 by stating that we live in a world where journalism is in deep crises and both misinformation and disinformation in mass media are growing day by day.
However, many scholars claim that the term 'fake news' is totally inadequate to describe the complex phenomena of "misinformation" and "disinformation". Moreover, in the theory, these words are used to identify the information as false; however, "misinformation" does not specify intent, whereas "disinformation" assumes the spread of false information to deceive2.
The term "online propaganda" is also used to describe exaggerated and false information intended to promote somebody or something. As for the term "rumor", one might consider non-confirmed information, which easily leads to the spread of misinformation. Similarly, the term "confirmation bias" characterizes the emergence of radically contradicting viewpoints on the same event (political) on SNSs. It is typical for users who interpret the political events or news from one side which in turn can be accepted and shared by others. Undoubtedly, this also can be a fresh start for the dissemination of distrustful information in social media. That is why, in many cases, the above-mentioned leads to confusion with "disinformation" and "fake news".
In SNSs, we encounter supposed "not reliable information" that is not at all disinformation. Because of the blurred classification of disinformation, it has become difficult to differentiate partially fake, manipulative or absolutely improper content. Theoretically, there are two types of disinformation; "Foreign-led disinformation" which aims to put efforts to manipulate public debate in another country and "Internal disinformation" (government/opposition), which is used by the governmental powers to manipulate people, attack opponents, gain mobility support, and by oppositional powers in order to blackmail the government. For sure, many types are out of these groups, however, in all cases, disinformation acts as "a weapon to people and against people".
According to J. Fetzer, disinformation includes dissemination of misleading, incomplete or inaccurate information with an intention to distort the truth (Fetzer 2004). Fallis defines it as a type of misleading information that can create false beliefs and can be propagated through various channels such as television, magazines and newspapers (Fallis 2015). In turn, states that "disinformation is about the destabilization of a society and a state through massive psychological conditioning of the population. It is a tool to pressure a state to make decisions that are in the interest of the opponent that is the aim." (Policy Department for Citizens' Rights and Constitutional Affairs 2019). Summing up the two above-mentioned definitions, we may bring them to one compound definition; "an intentional propaganda of fake news and distrustful information which is accompanied with the organized mass online and offline discussions in order to gain concrete political dividends".
2 Sargsyan, Lusine. 2020. "Fighting Fake News or Censoring Speech Online." EVN Report. Accessed December 10, 2021. https://evnreport.com/raw-unfiltered/fighting-fake-news-or-censoring-speech-online/.
In many cases disinformation is a result of the dissemination of not well-cultivated news or poor-quality information. Generally, the latter is the result of unprofessional work. This kind of disinformation can be easily revealed mainly by newly developed and significantly productive fact-checking digital tools used by professional investigators in journalism. Along with this, a profound number of social news portals, user accounts (including fake ones) are created or sponsored by political organizations to spread disinformation.
Disinformation is a crucial weapon during the interstate conflicts and tensed diplomatic relations as well. By creating fake news portals or by sponsoring the spread of fake news through the other countries' media channels, the conflicting counties are trying to destabilize the domestic social-political life of their adversary.
Often, for this reason, the media channels of regional or neighboring countries are widely used as they are accessible for the audience (society) they target. The last is massively implemented by Azerbaijan in the context of the Nagorno-Karabakh conflict, particularly during the Nagorno-Karabakh Second War took place in 2020. In what refers to Azerbaijan, the spread of fake news through hacked Armenian Facebook accounts and online channels has been assessed as Azerbaijani governmental actions to "create chaos" among Armenians both in Armenia and in Artsakh3.
The spread of disinformation is a step towards political fluctuation, as it boosts the feeling of doubt and distrust in society. In the case of traditional media (press, TV, radio), disinformation is not obvious and significant that much, as generally, before the publication the information is subjected to the multilevel filtration and cultivation. The latter allows checking the sources of information and its reliability.
Generally, Facebook acts as a mirror; it reflects the "quality" of the state and society. However, very often, the reflection is not clear. Gegham Vardanyan, Editor in Chief of Media.am, states that Facebook either sharpens existing problems or suggests fake agendas to distract people from urgent and real issues. It mainly depends on who the Facebook user is and what is the main goal of the usage4. Ashot Melikyan, the chairman of the Committee to Protect Freedom of Expression, states that there is no sign in situation improving and fake news, conscious alteration of facts, and manipulations continue to be widely circulated on Facebook.
The sources of disinformation dissemination
In the theory, a set of performers are mentioned as sources of mass disinformation dissemination. One of the well-known typologies can be found in Joshua A. Tucker and others research; In this sense the following typology; "Individual trolls", "hired trolls", "bots", "fake news websites", "conspiracy theorists", "hyper partisan media", "politicians", "foreign governments" is highlighted.
The term "troll" is used to describe people who bait others in order to evoke some response and reaction. This is exclusively done on purpose (Phillips 2015). Joshua A.
3 Horizon. 2020. "Artsakh official warns of Azerbaijani fake news in Armenian groups on Facebook." Accessed December 10, 2021. https://horizonweekly.ca/en/artsakh-official-warns-of-azerbaijani-fake-news-in-armenian-groups-on-facebook/.
4 A piece from the interview in the frame of the research; Media Initiatives Center, January, 2020.
Trucker discusses the term of "an individual troll", which might imply actions of political and apolitical nature. In contrast to this, "hired trolls" are paid by companies, politicians, political organizations, and other performers write fake posts and comments in public forums and groups (Mihaylov et al. 2015). Due to their illegal and harmful nature in terms of national and state security, they are trying to act secretly.
"Bots" are part of computational propaganda. They are software tools that produce international and local news and data, which are totally misleading and false. Hence, bots are used to manipulate people and politicians by affecting their discourses. Along with this, bots are also related to inflating politicians' "follower" and "like" counts (Woolley and Howard 2017). One might not underestimate the significant role of bots during the electoral period either to support or to attack candidates.
Another term is "fake news websites", under which specific outlets of profit-based activities should be mentioned. Fake news websites are producers and disseminators of disinformation on regular bases. The spread of news articles leads to the engagement of a large auditorium. Once users visit the news sources, they are directed to the original site where the whole content is located (Allcott and Gentzkow 2017). The influence of these kinds of web pages is substantially strong.
Next performer is "conspiracy theorist". The latter works on the production and dissemination of conspiratorial contents. It is a tool of widespread usage, especially in the case of "virtual political hostility" during the electoral period and global crises such as COVID-19.
"Hyper partisan media" is the primary incubator and disseminator of disinformation for today (Faris et al. 2017). In social media platforms, there are a variety of blogs and news pages that are designed to serve the interests of right and left wings of power and different partisan figures. SNSs become a virtual "battlefield" between them. The same research claims that hyper partisan news is much more widely shared on Facebook than the types of explicitly fake news stories described above. The actions of politicians in the frame of SNSs are much more predictable as they seek fame and recognition (Berinsky 2017). In this way, they may and, in many cases, they do justify all their formulated and adopted strategies. In this sense, the application of hyper partisan media acts as a supportive tool.
In Armenia the main disseminators of misinformation are so-called "Mushroom sites"5. These kinds of sites and digital accounts have a "short life" and they have low consistency. In the majority of cases, the users and accounts through which disinformation is being disseminated, have active digital behavior and are interconnected to each other. As a rule, those accounts are authored and regulated by the same person or a group of people. One obvious feature of such accounts is the absence of contact information and information about the account history. Moreover, the articles may have no author (or they are unknown), without any mark on the primary source. This reflects both the unreliability of a certain sources and the strengthening of a tendency due to which accounts are used to multiply clicks and thus lead to the spread of disinformation.
Finally, the research on the negative effects of disinformation on how foreign governments act is still an urgent puzzle to focus on. In this regard, the usage of
5 Ghazaryan, Karine. 2020. "Mushroom Sites: the Simple Business Model of Troublesome Journalism." Accessed December 10, 2021. https://media.am/en/verified/2019/02/15/11066/.
disinformation is applied to influence regional opinion and lead to the direction of "international wind". The recent years' studies and immense research and analysis are the follow up of the foreign relations of Russia, China, Iran, and Venezuela with Western countries, including the United States and vice versa (Vanderhill 2013; Nocetti 2015).
In this context, another major problem in Armenian media is the growing tendency of mass translations of international news without even professional checking of the credibility of the information. "Translated disinformation" considers the disinformation of sources such as dailies and online press which is much more attractive and "reliable" for people.
A. J. Berinsky claims that fluency is a powerful tactic in case of dissemination of disinformation (Berinsky 2018). This means that the content of the news should be clear and simple. The increase of fluency may enhance rather than reduce false beliefs. G. Pennycook and others, in turn, focuses on the importance of repetition as any false news which is on the headlines more than twice is more likely to be treated as accurate (Pennycook et al. 2020).
"Virality" or a "viral" model of dissemination is not so common but yet a used technique in online disinformation. The term is understood as growth in the diffusion of a certain message through person-to-person contacts similar to the spread of a disease or virus.
Many researchers claim that broadcasting is much more effective than peer-to-peer news transmissions (Goel et al. 2016). Interestingly, the majority of literature on the topic focuses on the written content, however, it is a fact that visuals are easy to remember, because their influence on people is much more significant than any written content. Consequently, the visual and audio mechanisms of news spreading are also effective and highly used tools (Stenberg 2007).
The Atlantic Council's Digital Forensic Research Lab6 which is considered to be world's leading hub of digital forensic analysts, formulated so called "4D's Disinformation Model" aimed at uncovering the disinformation, categorizing and looking for ways to eliminate the sources. According to Atlantic Council's Research Assistant in Caucasus, Givi Gigitashvili, disinformation is strong in quantity, the truth is one and it can be only in one source the other thousands are false. The Lab formulated "Dismiss", "Distort", "Distract" and "Dismay" categories for disinformation and they can be found not only in clear but also as hybrid forms.
One of the articles of the New York Times published by Adam Satariano on January 17, 2019, revealed Russia-linked misinformation on Facebook channels7. According to the article, misleading political content aimed to influence people in Armenia, Azerbaijan, Estonia, Georgia, Kazakhstan, Kyrgyzstan, Latvia, Lithuania, Moldova, Romania, Russia, Tajikistan and Uzbekistan.
In parallel, large-scale propaganda and an unprecedentedly aggressive information campaign assessed as mass disinformation and hate speech have been spread around the branch of U.S. billionaire George Soros's Open Society Foundations in Armenia. The
6 Atlantic Council. 2022. "Digital Forensic Research Lab." Accessed December 10, 2021. https://www.atlanticcouncil.org/programs/digital-forensic-research-lab/.
7 Satariano, Adam. 2019. "Facebook Identifies Russia-Linked Misinformation Campaign." Accessed December 10, 2021. https://www.nytimes.com/2019/01/17/business/facebook-misinformation-russia.html.
top headlines on Facebook are "Soros foundation is organizing clashes in Armenia", "They want to destruct Armenia", "Soros's foundation's office in Armenia must be closed", "The foundation's workers and people cooperating with them are foreign agents". "Their activities should be prohibited", "They want the Armenians to come out against the people of Artsakh (Nagorno-Karabakh)", "They want civil war and blood", etc. The crisis over the image of an Open Society Foundation's activities in Armenia has been broadly circulated by Sputnik Armenia. Interestingly, Sputnik Armenia has been identified by Facebook as part of the pro-Russian "Sputnik" which is totally controlled by the Russian government.
On April 5, 2019, Armenia's National Security Service (NSS) arrested an individual who "hid behind" a Facebook page called "Dukhov Hayastan Open Society." The account had populiarity with more than 2,000 followers. The page activities were qualified as manipulative. However, right after the arrest, a lot of politicians and civil society activists expressed concern saying that it poses a threat to freedom of expression in Armenia8.
"Any regulation should be determined by law. Moreover, the NSS's authority does not have control over the internet, fewer punishments for distributing fake news", wrote down the head of the Freedom of Information Center of Armenia, Shushan Doydoyan.
On January 5, 2020, NSS determined another fake Facebook account owned by a man that was named as Diana Harutyunyan. The account initiated a range of false information dissemination about ongoing political situations in and out of Armenia. The last fake publication was the following: "N. Pashinyan has congratulated US President Donald Trump on the assassination of Major General Qassem Soleimani". This kind of disinformation may cause a significant damage to the national security interests of Armenia.
The ways to struggle against disinformation: does it work?
The revelation of disinformation is a difficult procedure as it is a kind of "black propaganda" that is mainly hidden. It is a global challenge and many international field experts claim that only the comprehensive usage of artificial intelligence can overcome this threat (Constine 2016). Undoubtedly, the idea of the implementation of artificial intelligence is far from today's opportunities and resources, especially when we talk about developing regions and countries (Demartini 2019).
Many countries and international organizations all over the world have initiated an organized struggle against social media-based disinformation dissemination at the state level. Despite the fact that an adequate regulation is missing and each action mainly corresponds with the level of development and political culture of the certain country, self-regulation and policy actions have made some differences in the online environment. Many experts and field researchers claim that the legal regulations adopted by many countries are not effective. According to them, the use of rough methods and application
8 Khachatrian, Marine. 2019. "First Arrest Reported In Armenian 'Fake News Probe'." Accessed December 10, 2021. https://www.azatutyun.am/a/29864161.html.
of obvious pressure aimed at recording tangible results in a short time, are simply unable to beat the threat.
Due to the European Commission Action Plan against Disinformation, the Report from the Commission on the implementation of the Communication "Tackling online disinformation" and the Facebook Baseline Report on Implementation of the Code of Practice on Disinformation, Facebook, as a platform to host public discourse on topics of public interests, has been changed significantly.
In Singapore, the (Government of Singapore 2019) went into effect since 2019 as a decisive step to struggle against 'fake news'. Media companies that fail to comply the standards face a fine of up to $722,000. As for individuals, they might face fines of up to $60,000 or prison for up to 10 years.
German Bundestag updated the law on social media in Germany on June 30, 2017 (Deutscher Bundestag 2017). According to that, all the information, such as news and posts on Facebook, Twitter, and YouTube which will be considered as disinformation, fake news, and hate speech would be immediately removed and blocked. It is worth mentioning that the main emphasis was made on Facebook. Many congressmen and political activists claimed that Facebook removes only 39% of the hate content and fake information from the site which is an obviously low index in terms of reaching the mission.
Since October 1, 2017, the Bundestag also passed "Law about Facebook" (Facebook-Gesetz). Undoubtedly, this is an important and a promising step; however there are some shortcuts in the law, which do not allow fighting disinformation effectively. For instance, according to the law, only the suspicious content would be removed from Facebook and the account holder itself would not be accountable for publishing and sharing it. In this sense, there is a huge risk of restriction and abuse of freedom of speech as there is no clear differentiation between "own opinion and illegal content of own opinion."
In addition to this, after the adoption of the law, Facebook officially announced that democratic states should not make private sector responsible for disadvantages and obligations of the state and. Moreover, according to the above-mentioned announcement, it is not possible to struggle against disinformation in the case when the state authorities are not directly engaged in the process. This shows that there is an obvious disagreement between the state and the private sector based on their interests.
In France, the law "Manipulation of information" (Nationale 2018) on social media has been passed on November 20, 2018, to struggle against fake news during the electoral period, especially during electoral campaigns. In the frame of the law, a state report has been generated in which disinformation is labeled as "a major threat to democracy". The law considers also an implementation of a maximum control over political posts. Furthermore, Facebook and Twitter should uncover all the financial sources of paid political advertisements. Many media experts in France announced that it is not only illegal but also immoral to allow any council or body to decide which one from suspicious content in SNSs is better to remove or keep. The same procedure may be observed also in Russia and elsewhere.
Summing up the above-mentioned, we can claim that from a political prism, SNSs are platforms where "everybody is manipulated by everybody". For news portals, the formulation of the topic and of the appropriate content is a product of manipulative
influence exerted by the government, opposition parties and a society. For society, the digestion of any type of virtual news carries significant risk of being a victim of not reliable and professionally checked flow of information. Hence, they are manipulated by the news portal which has been already manipulated by other actors. The same repetitive situation is seen in governmental actions when demonstrating their "exclusive achievements" through online media as well as in opposition actions that put an effort to blame and discredit the government.
Hate speech or hostile rhetoric? Path to cynicism and populism: "The rise of hate speech is something which worries me very much. Why? Hate speech is a common scorch, a common evil. Hate speech is very detrimental to political culture. It kills democracy. In the countries that now embark on a new chapter of their history in terms of building political culture and democratic institutions especially, it is very important that the manifestations of hate speech are addressed, managed, prevented and combated"9.
Social networkings sites and hate speech
Another widespread problem in modern SNSs is hate speech along with cynicism and populism. According to (Cohen-Almagor 2011), hate speech is bias-motivated, hostile, malicious speech aimed at a person or a group of people because of some of their actual or perceived innate characteristics. Due to (Chetty and Alathur 2018), hate speech is any speech, which attacks an individual or a group with an intention to hurt or disrespect based on identity of a person. In other words, hate speech can be defined as "any publicly demonstrated slander and abuse which is against a person or a group of people and aimed at discredit their actions or insult their personalities".
Hate atmosphere on the political ground is a regular thing in SNSs when it comes to small and significant political fluctuations. The main problem is its systematic and uncontrollable characteristic.
It is won't be a mistake to claim that disinformation is one of the main sources of hate speech. Armenian Human Rights Defender Arman Tatoyan stated, - "The enhancement of the institute of fake users is rather worrying. Our observations have shown that intolerance, hate speech, offences are mainly being spread by fake users".
Virtual interaction gives an opportunity to feel freer and more courageous. This might be clearly recorded while observing people's digital behavior, particularly their posts and comments under political statements of politicians, political organizations, influencers and official accounts of the government, National Assembly. In the virtual area, people express ideas that may never be expressed in the case of face to face discussions. In addition, expressions of hate speech have become a trend that is used as a shortcut way to get instant popularity without putting in more effort. Very often, the borders between the rights of self-expression (freedom of speech) are blurred and its possible abuse entails the strengthening of the atmosphere of hate and intolerance in the
9 Mediamax. 2019. "Piotr Switalski: Hate speech kills democracy." Accessed December 10, 2021. https://mediamax.am/en/news/interviews/31939/.
society. That is the main reason why the freedom of expression is perceived as one of the causes to occur hate speech and the latter always opposes freedom of speech.
The impact of hate speech on people's lives is not clearly identified and defined as it depends on a variety of factors such as age, the content of information, on the level of involvement in virtual-social life, even on the quality of education and ongoing circumstances. In this context, the influence of hate speech can be categorized as "direct" and "indirect" (or mediated). Hate speech has the potential of disturbing social peace in that exposure to hate speech shapes attitudes and influences actual behaviors (Müller and Schwarz 2021).
SNS is one of the dangerous and uncontrollable ways to actualize hate speech. Andre Oboler states, - "The longer the hate content stays available in social media sites, the more damage it can inflict on the victims and empower the perpetrators. If you remove the content at an early stage you can limit the exposure. This is just like cleaning litter; it doesn't stop people from littering but if you do not take care of the problem it just piles up and further exacerbates" (OHPI 2014).
Due to the unprecedented speed of the internet and flexibility of online media, the hostile rhetoric reaches a large audience. Based on (Duggan and Smith 2016), most social media users think that it is very stressful to communicate with people with whom they disagree on specific issues. The problem is that these kinds of communications are full of offensive and incendiary language. Additionally, the research claims that the political actors are frequent targets of incivility and harassment.
Another research claims that the politicians who are targeted in this way are less inclined to adopt an engaging style which reduces social media's potential for open and interactive political deliberation (Theocharis et al. 2016). This kind of vitriolic interaction in social media considers being one of the main indicators of raising or strengthening polarization and reducing the possibility to reach productive communication. It is worth mentioning that along with this, politicians themselves are also responsible for both disinformation and hate speech in social media.
Often, the spread of hate speech carries the risk to encourage violence. The rise in terrorism, xenophobia, islamophobia and anti-refugee sentiments in many countries in the world (including European countries) is getting its high peak due to the new media opportunities resulting in the formation of tricks and techniques for media manipulation. In the case of political hostilities, the physical clashes in the face of civil war and mass disorders are common phenomena.
Messages with hate content
"The Monitoring of hate speech" project initiated by (Armenian Helsinki Committee 2019), allowed monitoring a lot of online media sources and evaluating their engagement in the spread and encouragement of hate speech in Armenia in the period of July 2018 -June 2019. Due to the report, all the cases were typified in two groups; "hate speech" and "dangerous or harmful speech".
On 16 December 2019, a member of Parliament S. Khandanyan suggested to establish a working group that will be specialized in formulating and implementing new reforms on hate speech elimination. In the interview, a member of Parliament M. Karapetyan stressed that freedom of speech does not allow the demonstration of any kind
of hate speech. She added that their mission is to resist hate speech by criminalizing and condemning it10.
226 Article of Armenian Criminal Code envisages criminal liability for national, racial, religion-based discrimination. However, there is no regulation on prohibition of hate speech also in laws of RA on "the guarantees of the activities of a Deputy of the National Assembly of the Republic of Armenia" and on "Public service". According to the laws, public servants and deputies must respect all the people with whom they communicate during the exercise of their obligations.
On 15 May 2020, a proposed draft to supplement the RA Criminal Code with Article 226.2 was signed by President A. Sarkissian. According to the draft supplements, the publicly justified violence or propaganda of violence and public calls for violence based on age, sex, race, ethnic and social origin, religion, political orientation, would be criminalized11. The offense is punishable by imprisonment for a term of 1 to 3 years.
A set of different announcements by state figures and politicians (words, phrases) have been qualified as "messages with hate content" addressed to specific actors (persons, institutions). Among them was Yerevan city Mayor H. Marutyan's preelectoral public speech on September 2, 2018, when he used a "white and black forces" phrase to demonstrate the ongoing situation in the country. The chosen phrasing was highly criticized by political parties and civil society representatives both in online and offline platforms. Prime Minister N. Pashinyan assessed H. Marutyan's phrasings inaccurate. According to him, each statements and political announcement must fit into the revolution of love and solidarity; therefore, it is better to speak about brighter colors that black and white". Later, H. Marutyan released a post on Facebook by stating, - "I am not a perfect one and I believe I could have chosen a clearer phrasing". Even now the "white and black" expression is perceived negatively by many.
The concern among civil society organizations and political was intensified because of another political
On January 26, 2020, H. Hartenyan, a member of the Yerevan City Council, who is also a member of "My Step" faction, posted a photo of S. Tovmasyan (daughter of Constitutional Court Chairman H. Tovmasyan) on Facebook with this description: "This is the Facebook account of Hrayr Tovmasyan's daughter. Folks, forward!". This was estimated as a manifestation of hate speech as it contained a "call for some action", which was strongly criticized by both opposition and ruling party members. Due to the evolved public discourse on this case, it is obvious, that such kinds of announcements are improper and should not be justified in any way, even if authorities are aimed at making changes in the post of the president of Constitutional Court.
After this incident, H. Hartenyan was resigned from his position. Later he shared a post on Facebook with this content, - "Dear compatriots, due to the situation that has
10 Union of Informed Citizens. 2019. "Criminalization of hate speech without restriction of freedom of speech." Accessed December 10, 2021. https://uic.am/9676.
11 The National Assembly of the RA. 2020. Criminal Code of the Republic of Armenia (Adopted 18.04.2003)." Accessed December 10, 2021. http://www.parliament.am/legislation.php?sel=show&ID=134-9&lang=eng.
been created, I apologize and believe my work in the Council of Elders of Yerevan is incompatible. Thus, I am dropping my mandate"12.
On Facebook and in media outlets one might find hatred speech also towards homosexuals, their supporting people and organizations. Hate rhetoric on this topic is typical in almost all traditional societies with a poor level of democratization of political culture. In this context, it is worth mentioning the case with the first registered transgender woman in Armenia, Lilit Martirosyan, who addressed a speech on discrimination and violence at the National Assembly of the RA. Immediately after the speech, she was attacked by an intolerant attitude (verbal and non-verbal) by several members of NA.
Populism as a booster of political hate speech
Another component related to hate speech is populism. A range of political figures and leaders are publicly accused of being "populists" as common practices such as falsity of the political arguments, manipulative character of electoral campaigns, information manipulation, data abuse, etc. have broken down disinformation into new hybrid subtypes containing also hate elements. The main targets of politicians are the daily and loyal consumers of social networking outlets. This is a fact, which is clearly perceived both by politicians and media actors. As the populism is easily playing with the emotions and visions of the people, hence the SNS is the best area for populistic activities.
Populism is viewed as a global political trend. There are strong debates till now on whether populism is a positive force or it is a force to destroy democracy. Populism may lead to serious changes and reforms in the country by arming people who feel "isolated" from society. It can alter agendas of ongoing political developments and unite different people around one idea or an inspiration. It is an effective tool during the political upheavals, revolutions as well as during in transition from an authoritarian regime to a democratic one. However, it does not mean that populism should accompany further social and political life as, at some point, it would become simply uncontrollable. It may result in the decline of the legitimacy of a ruling party or state authorities and promote the feeling of despair in society, even the eruption of a new wave of protests. Moreover, it may lead to "political tribalism" by discouraging a political compromise.
Populism is a unique political communication technique between politicians and people which comes in different forms. In the case of democratic consolidation, populism might become a critically dangerous game. Political cultures are getting poor and degraded with the pressing presence of populism in politics. Populism in politics serves as "rich food" for the opposition as well, as populism gives birth to a new generation of populists. It is also in the center of media discourse of post-revolutionary Armenia.
In the case of populism another aspect should be comprehensively discussed; populism by the media and populism through the media (De Vreese et al. 2018). In a modern world populism is coming to life not only by the people, but also by the social
12 Panorama.am. 2020. "Yerevan councilman Henrik Hartenyan to step down." Accessed December 10, 2021. https://www.panorama.am/en/news/2020/01/27/Yerevan-councilman/2229289.
media outlets. Without any doubt, the traditional media is engaged in different populistic actions, but from this perspective, new media resources are endless.
Summing up the abovementioned, we can claim that hate speech in social media is an indicator of highly polarized political sphere which is mainly encouraged by political disinformation on Facebook. Due to A set of cases shows that the usage of hateful rhetoric especially by political and state figures is a matter of serious concerns that may lead to political and ideological intolerance. The last is due to the flexibility of online media, that responsible for the hostile rhetoric to reach a large audience and somehow influence people's political attitude.
Methodology
The research uses qualitative research method by providing descriptive analyses (statistics) through the online survey and content analyses by focus group discussions. A cross-sectional survey among 384 representatives (sample size) of 18-35 age range was conducted. According to the database of the Statistical Committee of the Republic of Armenia, in 2019 the population of Armenia was 2,965,269, from which 813,289 people represent 18-35 age (officially young)13. 34 young people were engaged in the focus groups conducted in Yerevan, Vanadzor, Talin and Gyumri cities, 350 representatives were surveyed via online questionnaires.
The focus groups were organized in December, 2019 and January, 2020 with the coordination of local NGOs located in the above-mentioned cities. The survey was conducted in March and April, 2020 through the open call on Facebook. The key component of the research is to portray the attitude of the young people on social media's negative manifestations and their virtual "familiarity" with domestic politics and state authorities. In parallel, the connection between usage of Facebook and its influence on the political attitude of the young people have been revealed.
Summary of Findings: Focus group analyses
Focus group sessions found that the majority of participants is assured that they differentiate news fake from trustful news in social media, hence nothing can harm their behavior. Both in theory and in practice the capacity to differentiate trustful information from misinformation does not act as a guarantor for being protected. It is much more decisive how people react to them. Even the young people who realize that they face with the fake information, inclined to read and in some cases comment it. Reading, commenting and sharing the fake information are the three obvious indicators that people are influenced and the manipulation is succeeded. Yet, the influence of disinformation is not specific and distinguished. It is ought to be mentioned, that according to behavioristic approach, people are inclined to believe in everything, they want to believe, no matter it is fake or not.
13 The data has been provided by Statistical Committee of the Republic of Armenia (ARMSTAT) as a response to my request through E-Request portal on 24.02.2020.
Similarly, Facebook as a "political tool" is not yet properly accepted by them. Through the "snowball effect" online news and information reach out to the offline area among people who are not active in social media or do not use it at all (especially elderly people). These are mainly the consumers of traditional media news portals. The fact is that the diversity of news does not guarantee its reliability and yet the poor media literacy in society makes people "exercise attraction" to new digital sources of news14.
Focus group data also showed that the young people are convinced that Armenian media (both online and traditional) is not free from the state's influence and people are subjected to governmental manipulation. Moreover, social media is under the manipulative attack of ex-power and current opposition as well. The latter is trying to use all also virtual tools on the audience to emphasize and exaggerate the failures of the government by blaming it in as "national betrayal".
Interestingly, all the young respondents were convinced that the political hateful atmosphere and disinformation are simply provoked by both the ongoing political situation in Armenia and the continuous political struggle between ex-power, ruling power and opposition15.
Most of the participants considered Velvet revolution in Armenia as a stimulus in terms of activation of social media as a political tool16.
Overall, focus group analysis showed that young people both from Yerevan and regions of Armenia demonstrate high activeness in virtual communication through social media sites.
They encounter disinformation and hate speech in the media landscape on a daily basis. However, they claim that they try to not act as disseminators of fake news as well as promoters of hate speech.
Descriptive analyses (statistics)
The percentage of female participants was 59.70%; the number of males was 40.3%. The 34.3% of respondents was from 18-24, 31.4% from 25-30 and 34.3% from 31-35 age range. Based on study findings, Facebook leads the most used social networking sites' rank with 60.6%, Instagram with 25.6%, Twitter with 11.1% and other sites with 2.7%. The majority of survey participants represented Social sciences (50.8%) and Natural sciences (40.6%), after Formal sciences (5.5%) and Humanities (3.1%). The overall descriptive analyses are provided below through the diagrams (see Diagram 1-18).
14 "In order to highlight the reliability of the news, my mother used to say "I read it on Facebook" or "The news is from the Internet". She had better realize, social media does not a guarantor of trustful information. No need to believe in everything she read!", - mentioned one of the participants from the Talin focus group.
15 "Social media helps us to become more interested in politics and in activities of active citizens", - mentioned by one of the participants from the Vanadzor focus group.
16 "Since Velvet revolution I am assured that demonstrations, gatherings or social-political movements mobilized through Facebook groups and activist pages are "working tools" in Armenia and can promote significant changes", - mentioned one of the participants from the Gyumri focus group.
Diagram 1. Importance of elections for strengthening democracy in Armenia
(350 respondents)
Do you consider elections important for the consolidation of democracy in Armenia?
Yes 94.3% No 5.7%
Diagram 2. Active participation of the young in elections (350 respondents)
Do you consider important the active participation of the young in elections?
Yes 95.7% No 4.3%
Diagram 3. Virtual activeness ofpoliticians, electoral behavior and political interests
(350 respondents) Do you feel the virtual activeness ofpoliticians affects your electoral behavior and political interests?
Yes 38.9% No 61.1%
Diagram 4. Hate speech, disinformation and political behavior on Facebook
(350 respondents) Do you feel hate speech and disinformation on Facebook affect your political behavior?
Yes 36.0% No 64.0%
Diagram 5. Hate speech and disinformation on Facebook (350 respondents) Do you think hate speech and disinformation on Facebook impact political behavior of your friends andfamily members?
Yes 62.6% No 37.4%
Diagram 6. The influence of virtual activeness ofpoliticians (350 respondents)
Do you feel the virtual activeness of politicians affects the electoral behavior of your acquaintances?
■ Yes 62.0% No 38.0%
Diagram 7. The role of social media in politics (350 respondents)
Do you consider social media as a decisive tool in politics?
■ Yes 69.7% ■ No 30.3%
Diagram 8. Social networks and the process of democratic consolidation
(350 respondents)
Do you consider social media to promote the process of democratic consolidation?
■ Yes 83.1% ■ No 16.9%
Diagram 9. Readability of political news on Facebook (350 respondents)
Do you read political news on Facebook?
■ Yes 84% ■ No 16%
Diagram 10. Demonstration of political interests and orientation on Facebook
(350 respondents)
Do you demonstrate your political interests/orientation on Facebook?
■ Yes 36.3% ■ No 63.7%
Diagram 11. Interest on politicians' activity on Facebook (350 respondents)
Do you follow politicians' activities on Facebook?
Yes 72% ■ No 28%
Diagram 12. Disinformation and reliable information (350 respondents)
Do you differentiate disinformation from reliable information?
■ Yes 93.4% ■ No 6.6%
Diagram 13. Upcoming referendum on constitutional reforms (346 respondents)
Will you take part in the upcoming referendum on constitutional reforms?
Yes 69.4% ■ No 30.6%
Diagram 14. Early parliamentary elections, 9 December 2018 (348 respondents)
Did you take part in Early parliamentary elections in 2018?
Yes 69.0% ■ No 31.0%
Diagram 15. Preferences in participation in political life (528 responses)
What are your preferable ways to participate in political life?
i Demonstrations 16.1%
Elections 31.8%
Party membership 2.1%
Virtual activeness 24.3%
Neutrality 21.2% Other 1.5%
Diagram 16. Methods to check the reliability of news (475 responses)
What are your methods to check the reliability of news?
1 Look for other news portals 39.40%
Inspect the source of the news 48.0%
Ask for professional opinion 10.30%
Other 2.30%
Diagram 17. Reaction on disinformation on Facebook (357 responses)
How do you react disinformation on Facebook?
Ignore 49.0%
Only read 34.2%
Read and comment 13.7%
Other 3.1%
Diagram 18. Reaction on hate speech on Facebook (340 responses) How do you react hate speech on Facebook?
■ Ignore 43.5%
■ Only read 41.2%
Read and comment 11.2%
Other 4.1%
Conclusion and discussion
"If experience is a teacher, then the current American technological experience teaches us that privacy has little value, but our right to express ourselves is critical to individual fulfillment" (Swigger 2013, 593).
It is still unclear what specific effects and in what extent social media influence people's political knowledge and political behavior. Furthermore, still science has no scientifically proven "cause and effect" scheme between the virtual negative manifestations of social media and people's political behavior. Therefore, in this research we showed that disinformation and hate speech being not new but novel and rapidly transforming negative manifestations in social media undoubtedly harm people's perceptions, thinking and actions. What is more significant, we stressed that in majority of cases disinformation leads to hate atmosphere and vice versa.
It is should be mentioned that generally the "virtual component" has a growing role in people's life. Thereby, we can look for and find out serious changes in people's political behavior taking into account the fact that different individuals will react differently to the same stimuli in social media.
SNSs, such as Facebook, Twitter, Instagram, etc. continue to be attractive and demanded, most of all among young people. Young people live in a reality that is simply unimaginable without the Internet. It is an irreplaceable source of contemporary education and socialization. In most of cases, young people even do not realize how they can be influenced by a simple word, a claim, content of news in SNSs.
Young people develop their political knowledge, shape their personal political identity by studying activities of political parties, goals of movements, reading political news, following political-public figures and activists, discussing ongoing political developments and transformations, raising questions and sharing their opinions, etc. It is a construction of a strong and long "social chain" where the formation of each circle is decisive in the process of creating the political behavior of the young people. Moreover, young seek independence and the environment without any control (especially from parents, especially in conservative societies). SNSs are irreplaceable portals to achieve these goals.
The findings address that cyber disinformation and hate speech in Armenia are mainly provoked and encouraged by two general components; political and social. In the case of political factors, the ongoing domestic political situation and foreign influence interests are highlighted. Whereas, in the case of social factors, the level of media literacy and social media's raising role in society should be emphasized.
It is obvious that the domestic political situation in Armenia from 2018 to 2020 was a "hot pot" of interaction between three political actors; government, opposition, expower. The revolutionary government tried to accomplish systematic reforms in the country, which was accompanied by regular attempts to justify its political actions and initiatives. In addition, they aimed at strengthening their position and maintaining the legitimacy they gained during the revolution. The parliamentary opposition did its best to introduce itself as the "only real oppositional power" in the whole history of the Republic of Armenia and the opposition, to discredit a new government. All the three actors used social media as a tool to demonstrate themselves and gain supporters and all
the strategies on that way are accompanied by legal and illegal, accepted and not accepted, moral and immoral actions. In this way they promoted a hateful political atmosphere through the mass usage of disinformation and manipulation.
Summing up the overall analyses, the following recommendations are provided;
1. To suggest legal and comprehensive definition for "cyber disinformation " and "cyber hate speech" terms. Due to the missing coherent definitions, the above-mentioned terms are not only the objects of controversial interpretations and debates in the context of human rights and freedoms, but they are also subjected to the situational manipulation in SNSs. In addition, there should be formulated a distinguished categorization of hate speech and disinformation, as not all the content that encompasses a discriminatory word-food can be described as hate speech, and not all the information which encompasses untruthful information can be categorized as disinformation.
2. To review existing legislation on hate speech and disinformation dissemination (legal regulations). In order to improve the legislation, an evaluation and possibility of adoption of an advanced system of norms against hate speech/disinformation should be discussed which would be based on the combination of international experiences and cultural peculiarities. From this perspective new strategies should be formulated and targeted which would be based on the study of not only international best experiences but also failures. As research shows, some European countries examples failed with the adoption of legal regulations against the negative manifestations of social media mainly because of the blurred borders between the Right to Self-expression and Freedom of Speech and speech that can be considered as hateful. Any discourse on the adoption of a law on disinformation and hate speech in social media should lead to justified frames of the legal methods of punishment. More and more countries engage (or try to engage) legal words in the regulation of social media, but it tense the relations between the state and society and weaken the communication between them. From the other side, it challenges the possible cooperation between the state and private sector. Legislative regulations are strong if they are accompanied with the strong understanding of the critical problem and its irreversible consequences. The existing legal regulations are imperfect and should be modified based on many factors, for instance, on the level of development of new media as a part of social and political life, on the level of stability of civil society and, what is more important, on the peculiarities of political culture.
3. To promote long-lasting and productive cooperation between the Armenian government and other stakeholders (politicians, civil society (active engagement of the young people), journalists and private sector). All the actors engaged in media discourse should be ensured that they are allies and follow the same interests in this virtual battle. The latter might succeed in case of initiating targeted mass informative campaigns. In order to systemize and control their collaboration, an independent and free union or an initiative should be organized which will act as an "umbrella" for actors' joint actions. It is important to note, that the trust between the actors mainly depends on the state's (government) sincere willingness to cooperate and act transparently.
4. To promote the development and mass usage of online learning tools among people. New media should never be only a source for information, news and entertainment, but also a regular component of the contemporary educational system. There is a huge need to formulate alternative and applicable tools to raise civil education
on a higher level and try to seclude it from the political ambitions. In the case of Armenia, the best start can be the establishment of a hybrid educational system (traditional and modern). Without underestimating the productivity of traditional education in Armenia, there should be stated that the convenience and flexibility of international education can be reached only by contemporary modes of education, including non-formal education. What is more important, in the people's consciousness, education should never be associated with politics. The struggle against COVID-19 in Armenia proves that the necessity of online learning and teaching tools should not be applied only during force majeure situations, but should become a lifestyle. The research shows, that the negative manifestations of new media are the main reasons for people to get disappointed and lose the faith in it. The same situation, unfortunately, is seen in the case of NGOs in Armenia. Being a part of civil society and representing the rights and freedoms of people, NGOs are not wholly accepted first of all by people. Similarly, new media is also an inalienable part of civil society. Civil society is strong and capable if all the actors and organizations in it enjoy the support of society. New media should serve the interests and prosperity of each citizen. Only belief in the reliability of new media may help people take seriously the media's role. From this perspective the promotion of the online component in the educational system of Armenia is more than justified.
5. To promote media literacy and ethical media usage. It is not shameful to be a not media literate person. It is a global tendency and global challenge. "Media literacy"/"Cyber hygiene" should be included as subjects at schools and in high educational institutions. Armenia needs additional efforts to be put into the education of elder generation too. This also supposes the organization of regular and condensed trainings and pilot educational programs for employees by Media research centers, NGOs and initiative groups. People influence others with their opinions, knowledge and information they carry. It is important to use new media information carefully, as it quickly spreads not only online but also in the offline area by affecting others who confuse the "mediated information" analyzed by the provider with the real one. Both at regional and international levels, regular campaigns should be initiated in order to raise public awareness, educate society and empower people with the necessary skills of media literacy.
6. To newly evaluate the modern tendencies of political participation and direct it to the construction of democratic relations between society and state. From the one side, the high engagement of citizens in social media, from the other side, the high engagement of social media in politics, give us the confidence to state that social media can serve as an applicable tool to bridge society and state. The main problem is to transfer the virtual activeness from unproductive "slactive" behavior to the high and smart usage of electronic tools of governmental management and participation. For this reason, the international standards of various electronic systems and tools can be adopted and implemented aimed at providing government-private sector (Business) (G2B) and government-society (G2S) strong communication and collaboration.
Acknowledgements
The research project has become possible by the support of USC Institute of Armenian
Studies, in partnership with the Armenian Communities Department of the Calouste
Gulbenkian Foundation. The contents of this publication are the sole responsibility of
the author and do not necessarily reflect the views of the donors.
References
Allcott, Hunt, and Matthew Gentzkow. 2017. "Social Media and Fake News in the 2016 Election." Journal of Economic Perspectives 31 (2): 211-236.
Armenian Helsinki Committee. 2019. "Hate speech monitoring." Report. Accessed May 1, 2021. http://armhels.com/en/.
Berinsky, Adam J. 2018. "Telling the Truth about Believing the Lies? Evidence for the Limited Prevalence of Expressive Survey Responding." The Journal of Politics 80 (1): 211-224.
Berinsky, Adam J. 2017. "Rumors and Health Care Reform: Experiments in Political Misinformation." British Journal of Political Science 47 (2): 241-262. https://doi.org/10.1017/S0007123415000186.
Chetty, Naganna, and Sreejith Alathur. 2018. "Hate speech review in the context of online social networks." Aggression and Violent Behavior 40 (May-June):108-118. https://doi.org/10.1016Zj.avb.2018.05.003.
Cohen-Almagor, Raphael . 2011. "Fighting Hate and Bigotry on the Internet." Policy and Internet 3 (3): 1-26.
Committee, LIBE. 2019. "Disinformation and propaganda-impact on the functioning of the rule of law in the EU and its Member States." Study. Brussels: European Parliament.
Constine, Josh. 2016. "Facebook chose to fight fake news with AI, not just user reports." November 15, 2016. https://techcrunch.com/2016/11/14/facebook-fake-news/.
Demartini, Gianluca. 2019. "Users (and their bias) are key to fighting fake news on Facebook - AI isn't smart enough yet." September 22, 2019. https://theconversation.-com/users-and-their-bias-are-key-to-fighting-fake-news-on-facebook-ai-isnt-smart-enough-yet-123767.
Deutscher Bundestag. 2017. "Gesetz zur Verbesserung der Rechtsdurchsetzung in sozialen Netzwerken (Netzwerkdurchsetzungsgesetz - NetzDG)." https://dip.bunde-stag.de/vorgang/.../81582.
Duggan, Maeve, and Aaron Smith. 2016. "The Political Environment on Social Media." October 25, 2016. Pew Research Center. https://assets.pewresearch.org/wp-content/uploads/sites/14/2016/10/24160747/PI_2016.10.25_Politics-and-Social-Media_FINAL.pdf.
Fallis, Don. 2015. "What is disinformation?" Library Trends 63 (3): 401-426.
Faris, Robert and Roberts, Hal and Etling, Bruce and Bourassa, Nikki and Zuckerman, Ethan and Benkler, Yochai. 2017. "Partisanship, Propaganda, and Disinformation: Online Media and the 2016 U.S. Presidential Election (August)." Berkman Klein Center Research Publication 2017-6. Available at SSRN: https://ssrn.com/-abstract=3019414.
Fetzer, James. 2004. "Information: Does it Have To Be True?" Minds and Machines 14 (2): 223-229. https://doi.org/10.1023/B:MIND.0000021682.61365.56.
Goel, Sharad, Ashton Anderson, Jake Hofman, and Duncan J. Watts. 2016. "The Structural Virality of Online Diffusion." Management Science 62(1): 180-196. https://doi.org/10.1287/mnsc.2015.2158.
Government of Singapore. 2019. "Protection from Online Falsehoods and Manipulation Bill." https://sso.agc.gov.sg/Bills-Supp/10-2019.
Mihaylov, Todor, Georgi Georgiev, and Preslav Nakov. 2015. "Finding Opinion Manipulation Trolls in News Community Forums." In Proceedings of the Nineteenth Conference on Computational Natural Language Learning, 310-314. Beijing, China: Association for Computational Linguistics. https://doi.org/10.18653/v1/K15-1032.
Müller, Karsten, and Carlo Schwarz. 2021. "Fanning the Flames of Hate: Social Media and Hate Crime." Journal of the European Economic Association 19 (4): 2131-2167. https://doi.org/10.1093/jeea/jvaa045.
Nationale, Assemblee. 2018. "Proposition de loi: relative a la lutte contre la manipulation de l'information (Texte définitif)." 20 novembre 2018. https://www.assemblee-nationale.fr/dyn/ 15/textes/l15t0190_texte-adopte-provisoire.pdf.
Nocetti, Julien. 2015. "Contest and conquest: Russia and global internet governance." International Affairs 91(1): 111-130. https://doi.org/10.1111/1468-2346.12189.
OHPI. 2014. "Press Release: Launch of Online Tool to Combat Hate." November 10, 2014. https://ohpi.org.au/press-release-launch-of-online-tool-to-combat-hate/.
Pennycook, Gordon, Adam Bear, Evan T. Collins, and David G. Rand. 2020. "The Implied Truth Effect: Attaching Warnings to a Subset of Fake News Headlines Increases Perceived Accuracy of Headlines Without Warnings." Management Science 66 (11) (November): 4944-4957.
Phillips, Whitney. 2015. This Is Why We Can't Have Nice Things. Cambridge: The MIT Press.
Policy Department for Citizens' Rights and Constitutional Affairs. 2019. "Disinformation and Propaganda-impact on the functioning of the rule of law in EU and its Member States." Study. Brussels: European Parliament. https://www.euro-parl.europa.eu/RegData/etudes/STUD/2019/608864/IP0L_STU(2019)608864_EN. pdf.
Reporter ohne Grenzen. 2021. "Regulierung sozialer Medien." https://www.reporter-ohne-grenzen.de/themen/internetfreiheit/regulierung-sozialer-medien?gclid=Cj0KCQiA_JW0BhDRARIsANymN0bQHrY3zksvpqH7HTt0_1Vc 0qYKH6g0kChpLFiNkh-XwqNcG_23SgEaAv56EALw_wcB.
Stenberg, Georg. 2007. "Conceptual and perceptual factors in the picture superiority effect." European Journal of Cognitive Psychology 18 (6): 813-847. https://doi.org/10.1080/09541440500412361.
Swigger, Nathaniel. 2013. "The Online Citizen: Is Social Media Changing Citizens' Beliefs About Democratic Values?" Political Behavior 35 (3): 589-603.
Theocharis, Yannis, Pablo Barbera, Zoltan Fazekas, Sebastian Adrian Popa, and Olivier Parnet. 2016. "A Bad Workman Blames His Tweets: The Consequences of Citizens' Uncivil Twitter Use When Interacting With Party Candidates." Journal of Communication 66 (6): 1007-1031. https://doi.org/10.1111/jcom.12259.
Vanderhill, Rachel. 2013. Promoting Authoritarianism Abroad. Boulder: Lynne Rienner Publishers.
Vreese, Claes H. de, Frank Esser, Toril Aalberg, Carsten Reinemann, and James Stanyer. 2018. "Populism as an Expression of Political Communication Content and Style: A New Perspective." The International Journal of Press/Politics 23 (4) (October): 423438. https://doi.org/10.1177/1940161218790035.
Woolley, Samuel C., and Philip N. Howard. 2017. Computational Propaganda Worldwide: Executive Summery. Working paper 11. Oxford, UK: Project on Computational Propaganda. https://demtech.oii.ox.ac.uk/wp-content/uploads/sites/-89/2017/06/Casestudies-ExecutiveSummary.pdf.
