CC BY
97
Copyright © 2019 by Academic Publishing House Researcher s.r.o.
K " *
I
Published in the Slovak Republic Media Education (Mediaobrazovanie) Has been issued since 2005 ISSN 199-4160 E-ISSN 1994-4195 2019, 59(2): 258-268
DOI: 10.13187/me.2019.2.258 www.ejournal53.com
Polarized Political Texts: a Possible Way to Measure Their Manipulative Capacity
A.A. Kazakov a , *
a Saratov National Research State University named after N.G. Chernyshevsky, Russian Federation
Abstract
Within the realm of modern media education studies, there is plenty of scholarship on the tools journalists use to manipulate public conscience. Far less frequently, attempts are made to create mechanisms to quantitatively express manipulative capacity of media texts. One of the possible ways to measure manipulative efficacy of polarized political messages (both online and in print) is substantiated in this article. The author's approach is based on fifteen parameters each of which is subjected to quantitative measurement in relation to a certain media story. Those parameters are (but not limited to) evaluative statements within headlines and subheadings, balance (i.e. amount of page space devoted to an alternative view), sources of information, quantity of emotion-laden words and precedent names, degree to which main arguments are well-founded. The article also contains results of this evaluation system beta-testing conducted on the basis of twelve online pieces from two American and three Russian newspapers. Key methodological limitations of this approach are formulated, further ways to optimize it are laid down.
Keywords: polarized media text, manipulation, media, media linguistics, media literacy education.
1. Introduction
The need to ascertain whether media content is true, authentic, and nonpartisan is becoming increasingly urgent today. "Fake", "fake news", "media manipulation', "fact-checking", "post-truth", "hype", "staged video" - all these (and some other) notions have come into everyday use of not only scholars and experts in the fields of political communication, journalism, or media education, but ordinary folks monitoring mass media messages as well.
As a natural result, the changing public reality attracts heightened scholarly interest. Within both Russian and foreign academia, the number of research on this issue is constantly on the increase. Throughout the Western world, the focus of such scholarships seems to shift toward the areas of media education and media literacy. European and North-American educators are working out different methodologies for teaching people (primarily pupils and students) mechanisms of blocking excessive and potentially false media content, try to cultivate critical thinking skills and abilities to check media messages' credibility, authenticity of photo images, etc. (Adams, Hamm, 2001; Aufderheide, 1993; Cappello et al., 2011; Fedorov, 2019; Fedorov, Levitskaya, 2019; Mackey, 2007; Potter, 2008; Solik, Minarikova, 2014).
In Russia, this area of study has begun to actively develop fairly recently - roughly since the end of 1990s. Anton Chekhov Taganrog Institute, Moscow State University, and Higher School of Economics (National Research University, Moscow) are usually recognized as leading centers of
* Corresponding author
E-mail addresses: [email protected] (A.A. Kazakov)
- 258
media and information literacy research. In our country, the number of scholarly journals specializing on these issues is growing; conferences, educational schools, seminars, and workshops are held; articles and books are published; bachelors and masters programs on media education are set up.
2. Materials and methods
Having analyzed more than hundred media education studies that are the most often quoted in both Russian and foreign academia, I picked out those of them that raise an issue of media texts' manipulative potential. Then, special attention was given to rare attempts of measuring media literacy skills and competencies (Arke, Primack, 2009; Ashley et al., 2013; Potter, Thai, 2016). With the help of comparative method, existing approaches were juxtaposed and contrasted. The most meaningful and clear elements were then borrowed for my own approach.
I also gave careful perusal to scholarship on certain manipulative tools used by journalist in order to affect public opinion. As a result, I extracted those of them usually mentioned in this regard and put them in a central place of my own measuring technique.
At last, I bore in mind media literacy concept that served as an ultimate context for my research. After all, an ability to measure manipulative "charge" of media messages matters a lot to people in terms of their media education: if they know how to do it, they are less vulnerable to fakes, malinformation, and stuff like that. To that end, numerous definitions of media literacy were also examined (Fedorov, Levitskaya, 2016; Fedorov, 2015; Zhizhina, 2016). For the purposes of this study, I define media literacy as an ability to find information amidst a deluge of media messages, to critically interpret and analyze it, to check its credibility and - if necessary - to create their own short media texts.
3. Discussion
Different aspects of polarized political texts have been reflected by Russian and foreign researchers. To start with, it should be noted that the media effects field in general is in a rather controversial state now. Some scholars believe that mass communication's first paradigm, media effects, "is in a state of crisis rather than a preparadigmatic state or a state of normal science", born from its inability to make progress in answering questions about media effects (Lang, 2013: 11). While others argue that "the field is not in crisis, but has made impressive strides in answering media effects questions and explaining influences of media on different levels" (Perloff, 2013: 318). To my mind, each of these positions is only partly true. On the one hand, media effects paradigm is quite efficient in theorizing about the constantly-changing communication environment. In this sense, the Kuhnian paradigm-in-crisis model employed by Lang does not accurately describe the current state of mass communication research. On the other hand, there are still some methodological gaps within the scholarships. For example, no universal means of measuring media effects have been offered in the field so far.
Curiously enough, such ambivalence is true for both Russian and foreign studies. That being said, I still see the issues that are not properly elaborated within this sphere. A possibility to quantitatively measure media texts' manipulative potential is just one of them. By the term "manipulative potential", I understand the strength of a text's influence on the audience, its so-called suggestive efficacy.
Not so many efforts to measure media effects have been made so far. Manoliu and Bastien did it in relation to several series (House of Cards, The West Wing, and The Big Bang Theory) and their impact on political cynicism of their audiences. Results they got indicated that "series recognized for their intense negativity increase people's level of cynicism, while those portraying politics in a positive way do not have any impact" (Manoliu, Bastien, 2018: 547). No doubt, the very attempt to compare effects of different types of series deserves respect. However, only "more" or "less" terms can be applied here to manipulative potentials. As far as exact "weights" are concerned, no tools to measure them were offered.
Powers substantiated another angle to consider this. Through in-depth interviews, he examined how journalists from different types of local news organizations in one U.S. city with a diverse news environment define, measure, and discuss their work's impact. In particular, he measured impact in many ways, "including audience analytics and effect-oriented metrics such as audience awareness, public discourse, and public policy" (Powers, 2018: 460). Again, it is quite an interesting approach, but rather qualitative than quantitative one.
Of particular interest are also studies testing the mobilizing effect of conflict news framing in the context of electoral campaigns (Schuck et al., 2016) and predicting armed conflict by using newspaper text (Mueller, Rauh, 2018). Personally, I deem them to be very close to measuring media texts potential. Indeed, an ability to forecast voting behavior or conflict situations based on media texts, to some extent, has much to do with the "strength" of media influence. However, no exact algorithms to measure or calculate it were presented in research mentioned above.
A fair number of publications are devoted to the visuals' impact on the audience. Scholars investigate the effects of textual versus visual on assessments of politicians' competency and integrity, differentially for males and females (Boomgaarden et al., 2016). They also try to find out how visuals influence opinions and behavior (Powell et al., 2015) and how newspaper articles' layout style and text slant affect the perception of a newspapers' political orientation on the left-right axis (Schindler et al., 2017). No doubt, this strand of research contributes a lot into general comprehension of mechanisms used for affecting the audiences. Moreover, visual aspects of media texts are often neglected and this fact, in its turn, emphasizes the importance of such studies. At the same time, the very attention to visual appearances of textual information does not enable scholars to evaluate media texts' manipulative charge.
Apart from the lack of proper tools to compare and measure media effects, there are also some methodological problems. Potter is probably right arguing that "authors of these studies commonly select weaker design options over stronger ones" and "designers of most tests of media effects ignore the many theories available when designing their studies" (Potter, 2018: 5-6; Potter, Riddle, 2007: 96). Scharkow and Bachl discuss similar methodological concerns about measurement (Scharkow, Bachl, 2017). Supposedly, it is likely to be correct in relation to other segments of media effects field too.
Sometimes, literature focuses on measuring news exposure (Bartels, 1993; Liu, Hornik, 2016; Prior, 2009) or various effects of advertising (Freedman, Goldstein, 1999; Taylor et al., 2013). Clearly, the measurement of the ways people are exposed to media content is crucial for the understanding of media use and effects, even though it has been a challenge for a long time. However, I could not agree more with De Vreese and Neijens who wrote that "Today's media landscape, in which individuals are exposed to a diversity of messages anytime, anywhere, and from a great variety of sources on an increasing number of different media platforms, has complicated the measurement of media exposure even more" (De Vreese, Neijens, 2016: 74). Notwithstanding, interesting inferences are drawn. "When online advertising is added to a television campaign, the extra reach achieved is primarily duplicated" (Taylor et al., 2013: 200) and "exposure to negative ads appears to increase the likelihood of voting" (Freedman, Goldstein, 1999: 1189) are just two of them. Nevertheless, the tendencies and observations reported in such studies, alas, do not allow measuring media texts' manipulative "charge" itself.
Despite the fact that this kind of research attracts little scholarly attention (Potter, 2018: 2), I am fully convinced that, firstly, various media messages wield different manipulative power and, secondly, if one could measure it, they would be able to compare certain stories (and even editions!) and - ultimately - to work out recommendations on how to withstand manipulations more effectively.
It should be noted that a way to measure manipulative potential presented in this paper is primarily targeted for polarized discourse. Following van Dijk and Eissa, I deem it to be a kind of discourse originating from the conflict of political interests of several relatively big actors (including, but not limited to, states); as a rule, polarized discourse implies promoting narratives meant to smear opponents, and divides social environment into "us" and "them" (Dijk, 2008: 32; Eissa, 2014: 72).
As far as Russian scholarship on the area under consideration is concerned, attempts to describe and systemize manipulative methods used by journalists still comprise the bulk of it. Research projects conducted by Grachev, Melnik (Grachev, Melnik, 2007), Kara-Murza (Kara-Murza, 2015), Dotsenko (Dotsenko, 1997) have already become classics. Numerous ways to affect public conscience are given full consideration and accompanied by vivid examples in their studies. Worthy of separate attention are works by Dzyaloshinskiy (Dzyaloshinskiy, 2005a; 2005b; 2006), Mikhaleva (Mikhaleva, 2009), Danilova (Danilova, 2009), Dobrosklonskaya (Dobrosklonskaya, 2010), Hazagerov (Hazagerov, 2015, 2016), Skovorodnikov, Kopnina (Skovorodnikov, Kopnina, 2012). Thanks to all of them, the widest breadth of up-to-date manipulative techniques has been examined very carefully.
As we know, media texts have always been an important data source in political communication. However, in recent years, the feasibility of investigating large amounts of text quantitatively has changed. The Internet provides scholars with enough data, and the research community is providing accessible text analysis software packages, along with training and support. As a result, text-as-data research is becoming mainstream in communication. "Scholars are tapping new data sources, they are employing more diverse methods, and they are becoming critical consumers of findings based on those methods" (Wilkerson, Casa, 2017: 530). In this sense, Grimmer and Stewart seem to be absolutely right in their idea that "here lies the promise of automated text analysis: it substantially reduces the costs of analyzing large collections of text" (Grimmer, Stewart, 2013: 268).
Having analyzed numerous studies on that point, I feel safe to assert that attempts to measure manipulative component of media texts are made in very rare cases. One of such efforts was exerted by scholars from Krasnoyarsk - Kolmogorova, Gornostaeva, Kalinin, and Taldykina. They created a computer program capable of ranking the English language media texts about Russia according to their manipulative capacity. The scholars took into consideration such markers of manipulation as military and Nazi lexis, words with prefixes "pro-" and "anti", word root "soviet", references to the President of Russia V. Putin, etc. After computer had processed significant number of media stories, they divided all texts into four groups: articles without manipulation; articles with a low level of manipulation; articles with a medium level of manipulation; and articles with a high level of manipulation (Kolmogorova et al. 2016; 2017).
Samkova, a scholar from Chelyabinsk, offered another approach to evaluating manipulative potential of media texts. She found it feasible to derive a so-called manipulative power of misinforming messages from the quantity of pragmatically marked words and to link it to the content of readers' comments (Samkova, 2017).
I fully support the endeavors of the aforementioned authors. No doubt, their approaches deserve close scrutiny and active replication. Nevertheless, some limitations of the methods discussed are also worthy of note. The very fact of using military and Nazi lexis, words with prefixes "pro" and "anti", pragmatically marked notions as well as mentioning Putin by no means covers the whole range of tools for affecting public opinion. In fact, there are much more of them. Moreover, I am convinced that not only misinforming texts but also those not intended to send false information have a manipulative "charge". In my view, any media story - even the most fact-based and lexically neutral one - may well be fraught with manipulative potential.
4. Results
How can this manipulative potential be measured? I suggest using fifteen parameters each of which may serve as a marker of intended impact on audience's conscience. Any of these parameters can be expressed quantitatively. The sum of all markers' numerical values will therefore indicate the "scope" of manipulative potential of an exact text (with the caveat that the whole approach I offer can be applied to verbal media messages only - be it in print or online formats): the more points - the stronger manipulative capacity.
1. The first parameter - a heading and a lede (Vorontsova, 2017): in case they contain explicit evaluative assertion or a call to readers to do something (e.g. to vote, protest, believe, trust, buy, act, etc.), the text should get two points; if the evaluation and /or a call are conveyed implicitly -one point; no points - if there is no signs of influencing readers at all. It goes without saying that judging whether subheadings and ledes were expressed explicitly or implicitly would inevitably be to some degree subjective. Having said that, I am sure that in most cases the difference between these two options is quite obvious. See, for example, a clear attempt to make a judgement in The Washington Post heading "Trump's cynical attacks on the rule of law hurt the nation", implicit evaluation in Commersant heading "To live through the US elections", and no manipulative overtones in Rossiiskaya Gazeta heading "As a senator to a senator".
2. Photos, pictures, collages, cartoons, and caricatures - often with words written below them (Spodarets, 2015; Voroshilova, 2013): if they do not clearly correlate with the main content of the text or / and bear evaluation supporting the main idea of the story's author - one point; in case such evaluations and influence are conveyed indirectly - 0.5 point.
3. Balance and the way an alternative point of view is presented. To my way of thinking, every time an article is about complex and controversial problems, all main stances existing on the issue should be reviewed, including those opposing the author's own opinion. In terms of agenda-setting
theory (Weaver et al., 2004), it means the necessity to cover as many attributes of the second-level agenda as possible.
If there is no reference to an opposite view at all, it is worth three points; if an alternative view is not quoted and paraphrase accounts for less than twenty percent of the whole text - two points; if an alternative view is not quoted and paraphrase accounts for more than twenty percent of the whole text or if an alternative view is quoted but such quotations account for less than twenty percent of the whole text - one point. Why do I use a threshold of twenty percent for measuring this parameter? Based on my observations, bigger space is hardly ever devoted to discussing alternatives in modern mass media. However, standards of genuine journalism deem the balance (in general, without any numerical values) indispensable to maintaining objectivity of the press.
Also of note is that sometimes there may be no need for introducing an opposite view - for instance, in short news reports on what, where, and when has just happened. No points should be added in such cases.
4. Referring to the experts' opinions and using quotations (Chanysheva, 2017; Frolova, 2015; Krasovskaya, 2017). Every time when an author makes references to a person whose expertise is in the area other than that discussed in the text, 0.5 point is added. The same "penalty" - for every quotation that was taken out of context, not properly cited or distorted.
Note that unlike the first three parameters, this marker (as all that will follow) is cumulative: the same text may well contain several cases of this kind - therefore, the total amount of points got for such cumulative markers depends on the number of cases a scholar spots. Say, if in the text about politics (let it be Russia-US relations), a well-known artist (no matter Russian or American) was quoted three times and his or her words supported the author's stance, the final sum for this criterion would be 1.5 points.
Apart from that, I deem it necessary to take account of text length. Clearly, the longer the story, the more chances to find manipulative elements in it. This logic suggests that the longest media message will potentially be the most manipulative. To avoid this simplification (which I do not find accurate enough), I suggest to bring in a notional volume of one thousand words. If the test exceeds such length by a hundred words or less, points got for cumulative parameters number four - fifteen should be divided by 1.1; if there are 1200 words or less (but more than 1100), points should be divided by 1.2, and so forth.
5. Delivering facts that are not true - two points per each incident. In case there is a correction published within 24-hour period (for online editions) or in the next issue (in print) -one point.
6. The source of information (Grishaeva, 2017; Ivanova, Chanysheva, 2014; Panchenko, 2010; Suzdal'tseva, 2013). Every time the author makes reference to rumors, uses sources that do not place a high priority on documentary proof or factual precision (i.e. movies, novels, etc.), appeals to anonymous witnesses or insiders, the text gets 0.5 points. Assuming that in some cases such constructions are necessary, I believe that they may also lay down a smoke screen: hiding behind them, authors could voice ideas, concerns, suspicions, and versions that have no proof.
7. Putting words into opponent's mouth, ascribing thoughts and intentions to them - one point for each case. It is to be recalled that the way to measure media messages' manipulative potential I propose in this paper is primarily designed for polarized political texts. That is why the notion of "an opponent" is much of the time clear there. In American press stories about Russia, usual opponents are V. Putin, Russia as a political regime or, say, hackers accused of cracking the Democratic Party computers. In Russian stories about the USA, opponents may be the congressmen hostile to Moscow, political establishment as a whole, liberal mass media, and -sometimes - D. Trump.
8. Derision, mockery, sneering, as well as disrespect to national symbols, relics, national heroes or the people in general - two points for each case. I guess that such blatant disregard for the feelings of others should also be treated as manipulation: falling back on methods of this type, journalists implicitly affect consciousness, worldview, and mindset of the audience and set frames of reference conducive to their narratives.
9. Depicting opponents as intellectually or physically impaired: one point for each case, if it was expressed explicitly; 0.5 - for implicit allusions.
10. Using emotionally charged words and word combinations (i.e. labels, epithets, metaphors, euphemisms, comparisons, etc.). In my opinion, this is one of the most difficult (in terms of its revealing) markers of manipulation. Several scholars believe that the very fact of
employing such lexical tools indicates manipulative intentions of the author (Kovyazina et al., 2018; Mukhortov, Krasnova, 2016; Salakhova, Gracheva, 2016; Samkova, 2017). However, I admit the possibility that by doing so journalists may merely want to make their texts more vibrant, bright and vivid, having no political or manipulative agendas. Bearing this in mind, I feel compelled to impose a crucial limit on this criterion: all these emotion-laden lexical units should count only if they correspond with the author's view. Put it another way, if by using such words and phrases journalists intend to place somebody in a good light or, conversely, drag someone's name through the mire, this would be a sufficient reason to think of a deliberate decision (fueled by not only artistic motivation) to use such tools.
One may object that trying to determine whether an author is biased or partisan is rather tricky. I could not agree more: there is hardly ever a cast-iron guarantee that it is really so. In order to minimize an adverse effect of possible mistakes, it thus seems reasonable to use the following way of scoring: if the whole text (with due regard for a notional volume of one thousand words which was brought in above) contains from one to five cases of using emotionally charged words, it is worth 0.5 points; from six to ten cases - one point; from eleven to fifteen - 1.5 points, and so forth - 0.5 points for every five cases. Note that not all methods of expressing the vividness of media texts count here, but only those of them that aim to underpin the author's stance on the issue.
11. Mentioning precedent names or events. The same principle is applied: only those cases are taken into account that agree with the author's view. The way of scoring is the same, too. The only difference is an increment: as precedent names and events are used not so often (compared with emotion-laden lexis), 0.5 points will be got for every two cases. So, one or two cases are worth 0.5 points; three or four cases - one point; five or six - 1.5 points, and so forth.
12. Lack of proof, providing no evidence or argument for what an author states. This is another tool difficult to identify and prove. Following Tertychnyy, I refer different forecasts, predictions, assumptions, and anticipations to this group of manipulative techniques (Tertychnyy, 2002: 55), as well as imprecise, approximate, inaccurate, and hypothetical assertions of all kinds that, according to Suzdal'tseva (Suzdal'tseva, 2013: 42), lay the groundwork for manipulation. By way of illustration, appeals to the facts that are commonly known but not proved conclusively, mere assertions, arguments containing "highly likely", "odds / chances are", "rather", "seemingly", "apparently" are typical ways to achieve this goal. Each case of using one them is worth 0.5 points.
13. Hints, allusions, rhetorical questions, irony - again, only if they are used in a context advantageous for an author's narrative. 0.5 point for each case.
14. Praise given to opponents' critics or main rivals and - vice versa - criticism of their adherents or followers. I am sure that in doing so journalists affect their audiences, too. 0.2 points for each case of this type.
15. Quotation marks as an indication of irony or doubt. My own experience of beta-testing this approach suggests that one should differentiate between two possible reasons for using quotation marks. The first implies that it is done in order to show that someone's words are irrelevant, funny, or untrue. The second - when quotation marks are used before and after the words of an author - just to demonstrate their ambiguity and even absurdity. In the former case, depending on the context, it may be considered as either irony (see parameter number thirteen) or depicting an opponent as intellectually or physically impaired (parameter number nine). In the latter case, it should be deemed an independent tool concerning quotes, worth 0.1 point per each case.
Those were the main parameters that I find necessary to take into consideration while measuring the manipulative potential of a media text. An approach I offer was tested based on twelve stories - six American (three editorials of The Washington Post and three editorials of The New York Times) (Editorial Board a, 2017; 2018 a, b, c, d, e) and six Russian (two pieces from Kommersant, Rossijskaya gazeta, and Nezavisimaya gazeta) (Chernenko, 2018; Editorial Board, 2018, f, g; Novoselova, 2018; Zamahina, 2018; Zevelev, 2018). Each of them is an obvious example of polarized text as it deals with the current state of Russia-US relations.
I am fully aware that it is hardly possible to draw valid conclusions from the analysis of twelve media texts only. At the same time, some important moments are quite clear even now. For example, it turned out that parameters connected to headings / ledes, emotionally charged words, and lack of proof appeared to be the most popular ones. The majority of stories contained such markers of manipulation. On the other hand, I found no cases of derision, mockery, and
delivering false information. However, given the fact that the size of the sample was very small, I am far from eliminating these criteria from a scheme proposed. No doubt, provided the sample is much bigger, the markers mentioned will be encountered.
The mean aggregate score is 5.85. In other words, in total, after summarizing points got for all fifteen parameters, an average text received just under six points. Oddly enough, the mean manipulative potential of American stories appeared higher than Russian ones (6.05 against 5.65 respectively). I will repeat myself and say that is a little premature to make far-reaching conclusions based on this beta-testing. Nevertheless, there are some grounds for such a difference. The reason is that all American newspapers' stories were about a so-called "Russian meddling" into the US presidential campaign; hence a big number of emotion-laden lexical items, headings and photos jumping out at you, numerous "highly likely", "allegedly", "seemingly", etc. Articles of the Russian papers were a bit more moderate and neutral in this regard.
5. Conclusion
However, the main outcome is not about the mean number of points. It is about the fact that the approach I proposed proved to be fairly functional. In spite of the fact that it is far from perfect and needs to be polished, even in its current form, it allows to quantitatively express the manipulative charge of the media text.
As almost all other research tools, this approach has its drawbacks.
Firstly, any evaluation of texts' components is to some degree subjective. One scholar would consider an exact phrase to be emotionally charged, while another one may well find the same phrase neutral; one would think that a statement is well-founded, whereas another one would deem it to be a mere assertion, and so on. On the other hand, without any structured design, evaluations of manipulative potential risk to be even more arbitrary. In this sense, the approach I offer (despite its vulnerability) allows to play by the same rules and, what is also very important, to make comparisons across various media texts. Ideally, if the same text is read by two or more scholars (who preliminary got through an instruction), it is possible to minimize a degree to which their mean evaluation is subjective. Thus, when it comes to major studies, it is definitely worth trying.
Secondly, the whole approach suits principally polarized texts. It is hardly possible to apply it to "peaceful" media messages.
At last, one more obstacle should also be taken into consideration. The approach is intended for evaluation of text (in a narrow sense) stories and does not allow to consider nuances of television or radio broadcasts (i.e. peculiarities of background sound, light, and timing, to name but a few). Moreover, it cannot expose space-time features of print texts either.
That being said, no limits mentioned above are insuperable. If desired, the approach can be amended in order to move beyond text messages.
As far as the current beta version of the approach is concerned, I see the following first-priority ways to improve and refine it:
- making parameters of manipulation more clear and explicit; adding the new ones or exclusion of already existing are not beyond the realm of possibility;
- minor corrections to the value of each parameter (i.e. how many points should be accrued for every case of using manipulative techniques);
- further testing of the approach based on a much broader sample.
I am fully convinced that this kind of approach will pave the way for meeting new challenges existing in the fields of modern media education and political communication.
References
Adams, Hamm, 2001 - Adams, D., Hamm, M. (2001). Literacy in a multimedia age. Norwood, MA: Christopher-Gordon Publishers.
Arke, Primack, 2009 - Arke, E, Primack, B. (2009). Quantifying media literacy: development, reliability, and validity of a new measure. Educational Media International, 46 (1):
53-65.
Ashley et al., 2013 - Ashley, S., Maksl, A., Craft, S. (2013). Developing a news media literacy scale. Journalism and Mass Communication Educator, 68 (1): 7-21.
Aufderheide, 1993 - Aufderheide, P. (Ed.) (1993). Media Literacy: A Report of the National Leadership Conference on Media Literacy. Aspen.
Bartels, 1993 - Bartels, L. (1993). Messages received - the political impact of media exposure. American Political Science Review, 87 (2): 267-285.
Boomgaarden et al., 2016 - Boomgaarden, H., Boukes, M., Iorgoveanu, A. (2016). Image versus text: how newspaper reports affect evaluations of political candidates. International Journal of Communication, 10: 2529-2555.
Cappello et al., 2011 - Cappello, G., Felini, D., Hobbs, R. (2011). Reflections on global developments in media literacy education: bridging theory and practice. Journal of Media Literacy Education, 3 (2): 66-73.
Chanysheva, 2017 - Chanysheva, Z. (2017). Translation as a tool of ideological sabotage in intercultural political communication. Politicheskaya lingvistika, 5: 60-65.
Chernenko, 2018 - Chernenko, E. (2018). Russia and the US are further breaking apart. Kommersant, 22.05.2018. [Electronic resource]. URL: https://www.kommersant.ru/doc/3635736
Danilova, 2009 - Danilova, A. (2009). Manipulation by a word in mass media. Moscow: Dobrosvet, KDU.
De Vreese, Neijens, 2016 - De Vreese, C., Neijens, P. (2016). Measuring media exposure in a changing communications environment: Introduction. Communication Methods and Measures 10 (2-3): 69-80.
Dijk, 2008 - Dijk, TA. van (2008). Discourse and Power. New York.
Dobrosklonskaya, 2010 - Dobrosklonskaya, T. (2010). Issues of media texts analysis: experience of research on contemporary English media speech. Moscow.
Dotsenko, 1997 - Dotsenko, E. (1997). Psychology of manipulation: phenomena, mechanisms, and defense. Moscow: Moscow State University.
Dzyaloshinskiy, 2005a - Dzyaloshinskiy, I. (2005). Manipulative technologies in mass media (ending). Vestnik Moskovskogo universiteta. Seriya 10: Zhurnalistika, 2: 56-76.
Dzyaloshinskiy, 2005b - Dzyaloshinskiy, I. (2005). Manipulative technologies in mass media. Vestnik Moskovskogo universiteta. Seriya 10: Zhurnalistika, 1: 29-55.
Dzyaloshinskiy, 2006 - Dzyaloshinskiy, I. (2006). Manipulative technologies of mass media. Moscow.
Editorial Board, 2017 - Editorial Board (2017). Another cyberattack alarm is going off. We need to start paying attention. The Washington Post, 7.09.2017. [Electronic resource]. URL: https://www.washingtonpost.com/opinions/another-cyberattack-alarm-is-going-off-we-need-to-start-paying-attention/2017/ 09/07/ c93b7428-933c-11e7-aace-04b862b2b3f3_story.html?utm_term=.6f72c619e82c
Editorial Board, 2018a - Editorial Board (2018). No, it wasn't some hacker on a bed someplace. The Washington Post, 27.07.2018. [Electronic resource]. URL: https://www.washingtonpost.com/opinions/no-it-wasnt-some-hacker-on-a-bed-someplace/2018/07/27/d5d5c576-892c-11e8-a345-a1bf7847b375_story.html?utm_term=.740cc07e77d7.
Editorial Board, 2018b - By The Editorial Board (2018). How Do You Say 'Witch Hunt' in Russian? The New York Times, 13.07.2018. [Electronic resource]. URL: https://www.nytimes.com/2018/07/13/opinion/editorials/mueller-indictment-trump-russia-hacking.html.
Editorial Board, 2018c - Editorial Board (2018). Trump's cynical attacks on the rule of law hurt the nation. The Washington Post. 2.08.2018. [Electronic resource]. URL: https://www.washingtonpost.com/opinions/trumps-cynical-attacks-on-the-rule-of-law-hurt-this-nation/2018/08/01/298d0280-95bf-11e8-80e1-00e80e1fdf43_story.html.
Editorial Board, 2018d - By The Editorial Board (2018). Why Won't Donald Trump Speak for America? The New York Times, 16.07.2018.
Editorial Board, 2018e - By The Editorial Board (2018). Russia Attacks America's Election System. Trump Shrugs. The New York Times. 1.08.2018. [Electronic resource]. URL: https://www.nytimes.com/2018/08/01/opinion/editorials/russia-election-meddling-trump-putin.html?action=click&pgtype=Homepage&clickSource=story-heading&module=opinion-c-col-left-region®ion=opinion-c-col-left-region&WT.nav=opinion-c-col-left-region
Editorial Board, 2018f - Editorial Board (2018). Good will of the Russian and US presidents is not enough to improve relations between the states. Nezavisimaya gazeta. 29.07.2018. [Electronic resource]. URL: http://www.ng.ru/editorial/2018-07-29/2_7276_red.html
Editorial Board, 2018g - Editorial Board (2018). Trump - 4 percent. Nezavisimaya gazeta. 5.07.2018. [Electronic resource]. URL: http://www.ng.ru/editorial/2018-07-15/2_7266_red.html
Eissa, 2014 - Eissa, M. (2014). Polarized discourse in the news. Procedia - Social and Behavioral Sciences, 134: 70-91.
Fedorov, 2015 - Fedorov, A. (2015). Levels of university students' media competence: analysis of test results. Media Education, 1: 48-51.
Fedorov, 2019 - Fedorov, A. (2019). Schools and universities in audiovisual media: experts' opinions. Communication Today, 10 (1): 110-122.
Fedorov, Levitskaya, 2016 - Fedorov, A., Levitskaya, A. (2016). Comparative analysis of the indicators' levels of students' media competence development in the control and experimental groups. International Journal of Media and Information Literacy, 2(1): 16-37.
Fedorov, Levitskaya, 2019 - Fedorov, A.V., Levitskaya, AA. (2019). Synthetic media education model used in Commonwealth of Independent States (CIS). Media Education, 1: 30-36. DOI: 10.13187/me.2019.1.30.
Freedman, Goldstein, 1999 - Freedman, P., Goldstein, K. (1999). Measuring media exposure and the effects of negative campaign ads. American Journal of Political Science, 4 (43): 1189-1208.
Frolova, 2015 - Frolova, I. (2015). On expression of subjectivity in analytical articles of the British and Russian Quality press. Politicheskaya lingvistika, 1: 138-145.
Grachev, Melnik, 2007 - Grachev, G., Melnik, I. (2007). Manipulation of a person: organization, methods, technologies of information and psychological influence. Moscow: Institute of philosophy of the Russian Academy of Sciences.
Grimmer, Stewart, 2013 - Grimmer, J., Stewart, B. (2013). Text as data: the promise and pitfalls of automatic content analysis methods for political texts. Political Analysis, 21 (3): 267-297.
Grishaeva, 2017 - Grishaeva, L. (2017). "Tale of sense , if not of truth..." or on the principles of political media reality construction. Politicheskaya lingvistika, 4: 18-27.
Hazagerov, 2015 - Hazagerov, G. (2015). Generative model of propaganda: to the issue of gaps within cognitive theory. La identidad nacional a través del diálogo entre culturas. Research in the field of humanitarian sciences in the Ibero-American and Russian scientific space: 52-60.
Hazagerov, 2016 - Hazagerov, G. (2016). Teleological sense and social role of rhetorical figures' classification. Vestnik Rossijskogo universiteta druzhby narodov. Seriya: Lingvistika, 3: 89-102.
Ivanova, Chanysheva, 2014 - Ivanova, S., Chanysheva, Z. (2014). Rumor introducing discursive technologies in mass media political discourse. Politicheskaya lingvistika, 2: 39-49.
Kara-Murza, 2015 - Kara-Murza, S. (2015). Manipulation of the consciousness. XXI century. Moscow: Algorithm.
Kolmogorova et al., 2016 - Kolmogorova, A., Kalinin, A., Taldykina, Yu. (2016). Linguistic markers of manipulation in polarized discourse: parametric study. Politicheskaya lingvistika, 4:
194-199.
Kolmogorova et al., 2017 - Kolmogorova, A., Gornostaeva, Yu., Kalinin A. (2017). Computer program design for classifying English polarized political texts by their manipulative impact: results and discussion. Politicheskaya lingvistika, 4: 67-75.
Kovyazina et al., 2018 - Kovyazina, M., Il'yushchenya, T., Khabibullina, S. (2018). Negative evaluative language in description of anti-Russian sanctions policy and of Russian counter-sanctions in British online media discourse: a corpus research. Politicheskaya lingvistika, 2: 61-67.
Krasovskaya, 2017 - Krasovskaya, O. (2017). "Borrowed word" in the discourse of information war. Politicheskaya lingvistika, 6: 34-38.
Lang, 2013 - Lang, A. (2013). Discipline in Crisis? The Shifting Paradigm of Mass Communication Research. Communication Theory, 23(1): 10-24.
Liu, Hornik, 2016 - Liu, J., Hornik, R. (2016). Measuring exposure opportunities: using exogenous measures in assessing effects of media exposure on smoking outcomes. Communication Methods and Measures, 2-3 (10): 115-134.
Mackey, 2007 - Mackey, M. (2007). Literacies across Media. New York: Routledge.
Manoliu, Bastien, 2018 - Manoliu, A., Bastien, F. (2018). Does my favourite political television series make me cynical? Canadian Journal of Communication, 4 (43): 547-566.
Mikhaleva, 2009 - Mikhaleva, O. (2009). Political discourse: specifics of manipulative influence. Moscow: Libro-kom.
Mueller, Rauh, 2018 - Mueller, H., Rauh, C. (2018). Reading between the lines: prediction of political violence using newspaper text. American Political Science Review, 112(2): 358-375.
Mukhortov, Krasnova, 2016 - Mukhortov, D, Krasnova, A. (2016). Manipulation discourse markers within the framework of subjective evaluation. Politicheskaya lingvistika, 6: 120-125.
Novoselova, 2018 - Novoselova, E. (2018). Diamonds for General Eisenhower. Rossijskaya gazeta. 24.05.2018. [Electronic resource]. URL: https://rg.ru/2018/05/24/reg-szfo/kak-rossijskie-i-amerikanskie-arhivisty-sotrudnichaiut-v-epohu-sankcij.html
Panchenko, 2010 - Panchenko, N. (2010). Authenticity as a communicative term. Volgograd: Peremena.
Perloff, 2013 - Perloff, R. (2013). Progress, Paradigms, and a Discipline Engaged: A Response to Lang and Reflections on Media Effects Research. Communication Theory, 23 (4): 317333.
Potter, 2008 - Potter, W.J. (2008). Media Literacy. Los Angeles: Sage Publications.
Potter, 2018 - Potter, W.J. (2018). A review and analysis of patterns of design decisions in recent media effects research. Review of Communication Research, 6: 1-29.
Potter, Riddle, 2007 - Potter, W., Riddle, K. (2007). A content analysis of the media effects literature. Journalism & Mass Communication Quarterly, 84 (1): 90-104.
Potter, Thai, 2016 - Potter, W, Thai, Ch. (2016). Conceptual challenges in designing measures for media literacy studies. International Journal of Media and Information Literacy, 1: 27-42.
Powell et al., 2015 - Powell, T., Boomgaarden, H, De Swert, K. (2015). A clearer picture: the contribution of visuals and text to framing effects. Journal of Communication, 65 (6): 997-1017.
Powers, 2018 - Powers, E. (2018). Selecting metrics, reflecting norms: How journalists in local newsrooms define, measure, and discuss impact. Digital Journalism, 4(6): 454-471.
Prior, 2009 - Prior, M. (2009). Improving media effects research through better measurement of news exposure. Journal of Politics, 3 (71): 893-908.
Salakhova, Gracheva, 2016 - Salakhova, A., Gracheva, K. (2016). Linguistic analysis of manipulative means of mass media influence in the time of information warfare (on the material of German mass media). Politicheskaya lingvistika, 6: 132-138.
Samkova, 2017 - Samkova, M. (2017). The extent of a misinforming media text impact. Politicheskaya lingvistika, 5: 136-142.
Scharkow, Bachl, 2017 - Scharkow, M., Bachl, M. (2017). How measurement error in content analysis and self-reported media use leads to minimal media effect findings in linkage analyses: a simulation study. Political Communication, 34(3): 323-343.
Schindler et al., 2017 - Schindler, J., Kraemer, B., Mueller, P. (2017). Looking left or looking right? Effects of newspaper layout style on the perception of political news. European Journal of Communication, 32 (4): 348-366.
Schuck, Vliegenthart, De Vreese, 2016 - Schuck, A., Vliegenthart, R., de Vreese, C. (2016). Who's afraid of conflict? The mobilizing effect of conflict framing in campaign news. British Journal of Political Science, 46 (1): 177-194.
Skovorodnikov, Kopnina, 2012 - Skovorodnikov, A., Kopnina, G. (2012). Means of linguistic manipulation in the Russian press. Politicheskaya lingvistika, 3 (41): 36-42.
Solik, Minarikova, 2014 - Solik, M., Minarikova, J. (2014). Social recognition on a global scale: opportunities and limits of media reflection. Communication Today, 5 (2): 20-31.
Spodarets, 2015 - Spodarets, O. (2015). Persuasion potential of visual component in creolized texts of political media discourse (on the basis of US political discourse). Politicheskaya lingvistika, 4: 112-117.
Suzdal'tseva, 2013 - Suzdal'tseva, V. (2013). A verbal continuum "hypothetical character / falsity" in mass media discourse. Vestnik Moskovskogo universiteta. Seriya 10: Zhurnalistika, 4:
37-44.
Taylor et al., 2013 - Taylor, J., Kennedy, R., McDonald, C. (2013). Is the multi-platform whole more powerful than its separate parts? Measuring the sales effects of cross-media advertising. Journal of Advertising Research, 2 (53): 200-211.
Tertychnyy, 2002 - Tertychnyy, A. (2002). Genres of periodical press. Moscow: Aspect
Press.
Vorontsova, 2017 - Vorontsova, T. (2017). News title as an event representative. Politicheskaya lingvistika, 6: 21-25.
Voroshilova, 2013 - Voroshilova, M. (2013). Political creolized text: way to read it. Ekaterinburg: Ural State Pedagogical University.
Weaver et al., 2004 - Weaver, D., McCombs, M., Shaw, D. (2004). Agenda-Setting Research: Issues, Attributes, and Influences. In Handbook of Political Communication Research. Mahwah, NJ: Erlbaum: 257-282.
Wilkerson, Casa, 2017 - Wilkerson, J., Casa, A. (2017). Large-scale computerized text analysis in political science: opportunities and challenges. Annual Review of Political Science, 20:
529-544.
Zamahina, 2018 - Zamahina, T. (2018). Like a senator to a senator. Rossijskaya gazeta. 6.08.2018. [Electronic resource]. URL: https://rg.ru/2018/08/06/delegaciia-kongressa-ssha-posetila-rossiiu-vo-vtoroj-raz.html
Zevelev, 2018 - Zevelev, I. (2018). To pass through American elections. Kommersant. 6.08.2018. [Electronic resource]. URL: https://www.kommersant.ru/doc/3706462
Zhizhina, 2016 - Zhizhina, M. (2016). Media competence as strategic objective of media education: criteria of media competence. Media Education, 4: 47-65.
