Clarity for friends, confusion for foes: Russian vaccine propaganda in Ukraine and Serbia

---

Implications

Serbia and Ukraine are two of many countries around the world that are victims of extensive and effective Russian information operations related to COVID-19 vaccines. This issue has received an enormous amount of attention (Barnes, 2021; EU vs. Disinfo, n.d.; Gray & Edwards, 2020; Hyde, 2021; Kier & Stronski, 2021; Krainčanić, 2021; Kuzmanovic et al., 2021; Reyting Group, 2021; Sunter & Cappello, 2021; Thomas et al., 2020; UCMC, 2021). The magnitude of the problem makes it important in and of itself, but the circumstances also present a rare opportunity to understand Russian information strategy more broadly by directly comparing cases with the same topic and timing.

Harold Lasswell (1927) wrote that propaganda is directed at the ally, enemy, and neutral sides. However, research on modern Russian information campaigns almost exclusively focuses on the enemy. The conclusion of this prolific body of research is that the Kremlin’s uniform strategy is to sow information chaos, characterized by a lack of narrative consistency, to make truth seem subjective (Fitzgerald & Brantly, 2017; Global Engagement Center, 2020; Jankowitz, 2018; Singer & Brooking, 2018; Will, 2021).¹While focused on the domestic manifestation of this idea, one of the best books about this is Pomerantsev’s Nothing is true and everything is possible: The surreal heart of the new Russia (2015). Leaders of Russia have long operated under the assumption that information can be manipulated for political purposes; see Lenin (1902). 

This paradigm works well for understanding Russian vaccine propaganda in Ukraine, a country hostile to Russia. However, it does not apply to Russia’s information campaign in Serbia, whose public is pro-Russia. Russia may have a rich arsenal of divisive and anti-West narratives that it uses in other contexts in Serbia (Stiftung, 2018; Stronski & Himes, 2018), but on vaccines, Sputnik Serbia is taking the opposite approach to information chaos. It conducts its information campaigns through repetition of highly consistent pro-Russia narratives. Repetition, which psychological research shows to be highly effective, is a known component of Russian propaganda, but the finding that Russian narratives are consistent in Serbia contradicts prior research (Paul & Matthews, 2016).

This can be explained through the logic of the audiences’ receptivity to Russian propaganda, hinting at the Kremlin’s diverging motivations. In Ukraine, Russia is creating a package of pro-Russian ideas about geopolitics, domestic politics, and vaccines for its readers. If one part of the narrative resonates, the reader might be more open to a pro-Russian position on an unrelated topic included in the same article. In Serbia, where Russia can assume support from its readers on other topics, which can be addressed independently, vaccine messaging can be simpler. Additionally, attempts to polarize society through inconsistency and contradictions is a well-documented Russian technique, including in Ukraine (DiResta et al., 2018; Peisakhin & Rozenas, 2018). In majority-hostile publics, isolating and radicalizing receptive segments of the population helps achieve the Kremlin’s goal of increasing internal conflict. It would not be logical to adopt the same approach in a friendly country like Serbia, where the Kremlin benefits from maintaining a stable, status-quo political environment. 

This research has implications for what narratives journalists and civil society actors in each country should use to counter Russian propaganda, including public health NGOs, local independent journalists, and Western media outlets like Radio Liberty, Voice of America, BBC, and Deutsche Welle. It can also inform the government’s public outreach on vaccines, although these recommendations are unlikely to be adopted by the Serbian government, which is closely aligned with Russia (Kara-Murza, 2022). So far, targets of Russian information campaigns have struggled to proactively counter-narratives (Jones, 2018). This is a problem because the Kremlin prioritizes rapid dissemination of narratives, as people usually believe what they read first (Paul & Matthews, 2016).²The U.S. campaign to proactively combat information amidst the ongoing Russian aggression against Ukraine is a welcome change in approach (Merchant, 2022). However, these efforts still rely on knowledge of specific future information narratives gained through intelligence. This research allows for a broader, structural understanding of Russian propaganda that allows for a more proactive approach even in the absence of forewarning about upcoming Russian information campaigns. This research provides a better understanding of the structure of Russian propaganda narratives, not just the specific ones used currently, in different political contexts. This can help those fighting disinformation on vaccines and other topics proactively use their own narrative structures that are most likely to be effective. 

When Russia is trying to muddle narratives and conflate vaccines with other political issues, as in Ukraine and possibly other hostile contexts, the appropriate countermeasure would be to use clear and concise narratives focused only on vaccines. For example, journalists—including op-ed writers—should refrain from discussing the Russian vaccines alongside Russian military action in Ukraine, even if it is tempting to use both as examples of Putin’s geopolitical ambitions. Public health organizations and government officials responsible for pandemic outreach may want to avoid statements on politics entirely. This would help Ukrainians, including in the more pro-Russia separatist-held regions, develop a nuanced set of opinions about vaccines and other issues that are not bundled as strictly pro-Russia or anti-Russia, and could reduce polarization. The approach of addressing issues one at a time can be applied to counter other Russian information campaigns, including narratives about Putin’s current invasion of Ukraine. 

Serbia and perhaps other Russia-friendly contexts require a different approach. Adding nuance to issues that the Russian government is trying to frame as simple would dampen the impact of propaganda. While it would be unhelpful to take the Russian tactic in Ukraine of conflating only tangentially related issues—this would only fuel the fire, since Sputnik Serbia frequently comments on how the West likes to tie vaccines to geopolitics (RS #879, #989, #1533)³For ease of reference and to distinguish outside research from the random sample, articles in the random sample used for qualitative analysis are denoted by their number in the overall corpus for each outlet. RS refers to the Sputnik Serbia corpus, and UK refers to Ukraina.ru. Citations for sample articles are provided in the Appendix.—journalists and public health organizations and officials can provide context and caveats to Russian claims about Sputnik V and the Serbian government’s pandemic response. For instance, they could acknowledge data on the effectiveness of Russian vaccines, but also note the source of the data and compare it to alternative vaccines. Such additions may help even pro-Russia citizens moderate their views.

This study shows that further data-driven research is necessary to combat Russia’s information campaigns proactively. This issue has become even more urgent since the Russian invasion of Ukraine; initial reports have already shown how the Kremlin is tailoring its propaganda, including for the friendly domestic audience and hostile Ukrainian audience (Miburo, 2022). There has been some research on how Russia targets different narratives in various countries, including Ukraine and Serbia (U.S. Senate, Committee on Foreign Relations, 2018). However, such research does not address structural variation of information campaigns, which limits the ability of journalists and others to effectively counter it even before specific Russian narratives emerge. To my knowledge, there are only two other studies that address whether countries differentiate propaganda narratives used on adversaries and allies (Golan & Viatchaninova, 2014; Melki & Jabado, 2016), so these findings should be tested in other contexts. It is also critical to understand how tactics and platforms targeting specific audiences, not just narratives, differ when used on allies versus adversaries. This paper addresses what counter-narratives to Russian propaganda should be, but not how, where, and to whom they would be most effectively deployed, a subject for future research. 

Findings

Finding 1: Vaccine coverage in Russian propaganda incorporates a broad range of topics and is inconsistent in Ukraine but is narrowly focused and consistent in Serbia.

Topic models, which use quantitative methods to group articles into topics and reveal the most frequently used words in those topics, show that the range of Ukraina.ru articles about vaccines is wider than that of Sputnik Serbia (see Figures 1 and 2). Both outlets covered domestic vaccination campaigns and Sputnik V, but these were the only two topics in Sputnik Serbia reporting. Ukraina.ru incorporated vaccines into domestic politics to such an extent that it formed its own topic, which included opposition politicians like Poroshenko and Medvedchuk. In contrast, not even the Serbian president was in the top 40 words of either topic. Ukraina.ru articles about vaccines also mentioned geopolitics far more often, forming two additional topics. Qualitative analysis confirmed that vaccines featured in Ukraina.ru articles on issues from censorship to war in the Donbass, while in Sputnik Serbia, articles including the word vaccine were almost entirely about vaccines. 

Qualitative analysis showed that even in a small random sample of Ukraina.ru articles, coverage is often contradictory, except where Russia’s key interests – Sputnik V and the Donbas – are concerned. This includes whether Ukraine should use AstraZeneca (UK #808, #900), when Ukraine would likely receive Western vaccines (UK #1107, #1374), praise and criticism of China (UK #1004, #274, #37), and the varying assessments of the Ukrainian government’s vaccination efforts (UK #309, #1275, #1107, #1004, #1080), although the latter was mostly negative. Qualitative analysis did not reveal any contradictions in Sputnik Serbia. For example, all articles about China in the Sputnik Serbia sample reiterated that the government was excited to get Chinese vaccines and made the right decision to use all available vaccines (RS #1612, #1949, #2131), contrasting with mixed messaging on China in Ukraina.ru.

Finding 2: Russian vaccine narratives in Serbia are almost exclusively positive or neutral coverage of Sputnik V and the Serbian government’s vaccination campaign.

Only two clear topics emerged in Sputnik Serbia’s vaccine coverage (see Figure 1). The first is the Russian vaccine Sputnik V. Over half the articles are on this topic, and Serbia is not even one of the top forty words, so most articles regarding vaccines in Sputnik Serbia are not about Serbia at all (see Figure 3). A primary narrative is Sputnik V’s widespread and early implementation, revealed by words like already, first, register, early, and approve (RS #417, #1908, #989). A related narrative is that Sputnik V is safe and effective (RS # #417, #1451, #1908, #634). These claims are often overstated and incomplete, and they lack comparisons to the effectiveness of other vaccines (Nogrady, 2021).⁴One of these claims, namely that Russia is the only country to export vaccine technology, is disinformation. AstraZeneca is produced in multiple countries, which started well before the publication of RS #634. Several articles focused on how Russia is generous with Sputnik V around the world and does not have ulterior motives, as the West claims (RS #1533, #1908, #989); related words in the model are delivery, production, and world. These positive narratives about Russia and Sputnik V were much more frequent than in Ukraina.ru. Sputnik Serbia mentioned Russia 36% more and Sputnik V over three times more than Ukraina.ru.⁵Combined with all forms, such as “Russian,” “Russian Federation,” etc. This is calculated relative to the overall word count. p < 0.001 for both.

Finding 3: Russian vaccine propaganda in Ukraine also covers Sputnik V positively, but additionally incorporates geopolitics and criticism of the Ukrainian government.

Like in Sputnik Serbia, much of the vaccination and pandemic topic was useful, factual content with words like announce, report, day, data, case, dose, and Ministry of Health (see Figure 2; UK #879, #309). Ukraina.ru editors likely intended to lure readers into thinking Ukraina.ru is an objective outlet so they are more likely to believe the less reliable articles, a technique taught explicitly to propagandists during the Soviet era (Yakovlev, 2015). It is striking that this technique’s target of 60% neutral coverage precisely matches the percentage of articles in this topic (see Figure 4).

The domestic politics topic, which did not exist in Sputnik Serbia coverage, was quite distinct in Ukraina.ru, demonstrated by words like authorities and political, and the names of politicians. Qualitative analysis showed that many articles, including those that blended domestic politics with the vaccination and pandemic topic, criticized the government’s procurement policies: its refusal of Russian vaccines, use of subpar AstraZeneca and Chinese vaccines, and failure to obtain Western vaccines in a timely fashion (UK #44, #900, #1374, 1107, #1004). Articles in this topic also brought vaccines into other domestic issues, including President Zelensky’s unpopularity and repression of critics (UK #1080, #1004).

The two international politics topics, comprising about a quarter of coverage combined, incorporated vaccines into wide-ranging discussions of global issues, as demonstrated by the top positions of Russia, U.S., Biden, China, Donbas, Crimea, and UN (see Figures 2 and 4). Notably, the Ukraine-Russia-U.S. topic highlighted war in the Donbas, both prominent words in the topic. Qualitative analysis showed that the main narrative was that Russia is generous with its vaccines, while the Ukrainian government and the West do not care about Ukrainians’ health (UK #415, #1418, #1107), for example: “In the uncontrolled territories [i.e., the Donbass] people are receiving the vaccine, while the Ukrainian elites essentially threw the people to the whims of fate and themselves got vaccinated secretly in private clinics” (UK #1219). The diversity of the geopolitical themes to which Ukraina.ru ties vaccines differs substantially from the closest equivalent topic in Sputnik Serbia, Russian vaccines, which is limited to praise of Sputnik V and Russia’s generosity distributing it.

Methods

This study seeks to understand how Russia’s approach to propaganda differs when directed at the hostile versus friendly publics. This includes the key narratives and structure of the overall coverage. Ukraine and Serbia serve as good case studies because they are both European, post-communist countries but are polar opposites on the independent variable. Multiple recent head-to-head opinion polls show that the vast majority of the Ukrainian public is anti-Russia, and the vast majority of Serbians are pro-Russia (Hosa & Tcherneva, 2021; Mitchell, 2017). Most directly relevant, the Hosa and Tcherneva (2021) study found that 82% of Ukrainians do not trust Sputnik V and 75% of Serbians do.

The Russian propaganda outlets chosen for this research are Serbian-language Sputnik Serbia and Russian-language Ukraina.ru. Both are subsidiaries of the same Russian government-owned and operated media conglomerate, Rossiya Segodnya (Vedomosti, 2014). Therefore, they are not only both unambiguous mouthpieces for the Russian government but are also led by the same actors within the Russian government, so their narratives are directly comparable.

These sites are appropriate for analysis of Russian propaganda in Ukraine and Serbia because they specifically target each national audience and are influential, albeit in different ways. Ukraina.ru was launched in May 2014, shortly after the annexation of Crimea (Vedomosti, 2014). Therefore, it was explicitly designed to operate in a hostile context. Ukraina.ru propagates fake news stories that appear in Ukrainian outlets and social media, as identified by the Kyiv-based fact checker StopFake (n.d.).⁶The outlet publishes in Russian, which the vast majority of Ukrainians understand, even if they prefer not to speak it (Matviyishyn, 2020). The Belgrade-based, Serbian-language Sputnik service began in 2015 (Vučićević, 2016). Previous studies have shown that it wields significant influence through partnerships with local media (Brey, 2018; Jonsson, 2018; Stronski & Himes, 2019). Sputnik Serbia and Ukraina.ru play very different roles in the Russian propaganda apparatus namely because of the political differences that motivate this study; by design, there are no outlets in both countries with directly comparable tactics of influencing the public. Further research on the precise audience of these outlets and the way they integrate with the broader propaganda apparatus is needed.

The period of analysis, from December 1, 2020, to November 30, 2021, captured one year since the vaccination campaign began (Delauney, 2021; Ukrainskaya Pravda, 2021). All articles containing the word vaccine⁷As determined by a search result for that word through each website’s own search function. in this period were scraped from the websites using Python (2,417 articles in Sputnik Serbia and 1,552 in Ukraina.ru). I converted the words to their dictionary form so that all grammatical forms were considered the same word.⁸For this lemmatization process, I used the Python packages Pymystem3 for Russian and Classla for Serbian. Phrases of up to four words were created such that words typically appearing next to each other, such as Sputnik V, were considered one term.

Quantitative analysis was conducted using the statistical software R. This includes Latent Dirichlet Allocation (LDA) topic modeling, which creates topics based on the frequency with which words appear in the same text and ranks the most common words in each topic. Human judgement was used to decide on the number of topics that made the model most interpretable. To test and elaborate on quantitative findings, a small sample of twenty articles for each country was qualitatively analyzed. An equal number of articles about each topic was selected randomly from the group of articles within that topic.⁹16 of these were divided evenly between the topics and were selected randomly from articles highly representative of the topic, as the model estimating that over 90% of the content in the article is that topic. After meeting the 90% threshold, the sample is random. An additional four were chosen from articles containing a mix of topics to see how narratives may overlap. Mixed was defined as the model estimating that the topic with the largest represented topic does not comprise more than 60% of the topic.