A story of (non)compliance, bias, and conspiracies: How Google and Yandex represented Smart Voting during the 2021 parliamentary elections in Russia

Implications

The Smart Voting project was initiated by the currently jailed Russian opposition leader Alexei Navalny and his associates (Team Navalny) in 2018. Its aim is to undermine the political monopoly of the ruling United Russia party by preventing the dispersion of opposition votes across multiple candidates through voter coordination. The effectiveness of this strategy has been demonstrated, for instance, during the 2019 local elections in Saint Petersburg (Golosov & Turchenko, 2021). Via a dedicated website and the mobile app, Smart Voting shows voters which alternative candidate is most likely to defeat the candidate of the ruling party in their polling district. In the lead up to the parliamentary elections (September 17-19, 2021), the Russian authorities sought to limit the effect of Smart Voting: access to the app was restricted by the Russian telecommunications regulator Roskomnadzor on August 23, and the Smart Voting website was blocked on September 7. 

The Russian authorities also put pressure on Western and domestic online intermediaries to undermine the effectiveness of Smart Voting. Google and Apple complied with a government request to remove the Smart Voting app from their app stores (Lokot & Wijermars, 2021). At the same time, the Moscow arbitration court requested the two leading search engines in Russia, Yandex and Google, to remove Smart Voting from their search results. The request was presented as an interim measure in a case filed by the Russian company Woolintertrade, which had registered Smart Voting as its own trademark in July. Specifically, the court ruled to “Prohibit [Google/Yandex] the use of the ‘Smart Voting’ designation in search results of a search engine owned by the defendant [Google/Yandex] as one of the search keywords” (Elektronnoe pravosudie, 2021a, p. 1; 2021b, p. 1). How exactly the search engines should have implemented this order was not specified. 

The growing pressure on both Google and Yandex ahead of the 2021 elections highlights the increasing importance the Russian government attributes to expanding its influence over online intermediaries, including search engines. While some years ago the Russian authorities aimed to increase their control over online information distribution by (unsuccessfully) trying to create a state-controlled and a state-owned search engine (Sanovich et al., 2018), by now their strategy has shifted towards putting pressure on existing platforms to make them comply with the regime’s demands. In recent years, Russia’s capacity to exert control over online information distribution has been strengthened through increasingly restrictive Internet regulation and enhanced state control over internet infrastructures (Ermoshina & Musiani, 2017; Sivetc, 2021; Stadnik, 2021). 

Since Yandex is a Russian domestic corporation, it is generally thought to be more responsive to government pressure (Daucé & Loveluck, 2021) while Google is seen by many Russian NGOs as a protector of civil liberties (Bronnikova & Zaytseva, 2021) Compared with Google, Yandex was more intensively targeted by Russian regulatory mechanisms (Wijermars, 2021) and demonstrated more politically biased performance during periods of political contention (Kravets & Toepfl, 2021). However, the Russian authorities also put increasing pressure on Google as demonstrated by the growing number of requests for removing results from its search, particularly since the second half of 2020 (Figure 1). 

Across all Google services, Russia is responsible for the highest number of removal requests (Google, 2021). While more than 90% of these requests are related to copyright issues, it is impossible to estimate what share of copyright requests might be politics-related. Furthermore, Google is increasingly complying with the Russian government’s requests (Figure 2). This trend seems to be specific to Russia: in the case of Turkey, for example, Google’s compliance rate is around 40%, while for China it is close to 0% (Google, 2021). Given the limited information, it is difficult to ascertain whether the high compliance rate indicates a change in Google’s treatment of Russian removal requests (i.e., becoming more likely to comply with them) or a qualitative change in these requests (e.g., a higher proportion of evidently illegal content which is requested to be removed).

Against this backdrop of growing governmental pressure, we looked at how Google and Yandex dealt with the above-mentioned court order and found that both search engines continued to display results for the “Smart Voting” query. However, Yandex did limit access to the opposition’s web resources dealing with Smart Voting, which, presumably, was the main aim of the politically motivated court case. By contrast, Google did not remove Smart Voting from its search results and continued to link to Team Navalny resources. These differences in (non)compliance were amplified by political bias in the outputs, with Google outputs including more pro-Smart Voting content (and vice versa for Yandex), as well as Yandex tending to promote conspiratorial claims about Smart Voting coming from the Russian authorities (e.g., it being a project of the Pentagon aimed at harming the Russian people) and Google usually omitting such claims from its top results.

There are several implications of our observations. First, the example of Yandex illustrates the importance of scrutinizing how compliance with government removal requests as well as government-driven content moderation is implemented. Particularly in the case of “informational autocracies” (Guriev & Treisman, 2020, p. 1) in restricted media systems, such as Russia, where the preservation of the regime is largely contingent on successful censorship, the selective removal of links from search results can be used to facilitate dissemination of misinformation and anti-Western conspiracy theories. It can also serve as a form of “masked censorship” (Makhortykh & Bastian, 2020, p. 14), namely a less intrusive way of filtering out undesired information. The potentially damaging effects extend far beyond the removal of the links themselves as information suppression can erode democratization processes (Stoycheff et al., 2018). 

Under these circumstances, the decisions of online intermediaries concerning the implementation of removal requests play a key role in mediating the impact of censorship. Depending on these decisions, intermediaries might limit the visibility of outlets spreading extremist or false claims, but also facilitate state censorship by integrating it with “private censorship” (i.e., censorship effort by private companies) (see Beazer et al., 2021; Crabtree et al., 2015) or amplifying self-censorship practices (e.g., of Russian journalists in the aftermath of the annexation of Crimea) (Schimpfössl et al., 2020; Zeveleva, 2020). This raises questions about the feasibility of one-size-fits-all regulation of intermediaries’ activities that would be applicable to their operations across democratic and non-democratic contexts, especially in the case of countries with long-standing censorship traditions (e.g., Russia and many other post-Communist states) (Ognyanova, 2019). 

Second, our observations demonstrate the possibility of profound information inequalities forming between the users of Russia’s two largest search engines. While the mere presence of such inequalities is not unexpected, especially in the context of the increasingly fragmented and polarized Eastern European digital ecosystems (Urman, 2019; Urman & Makhortykh, 2021a), the empirical evidence of their amplification by algorithmic mechanisms is concerning. Considering that these inequalities in the case of Smart Voting (and, potentially, other political matters in the region) are subjected to political bias, such amplification may encase Yandex and Google users into platform-wide filter bubbles (Pariser, 2011), in particular as the Russian search market is roughly split between Google and Yandex (Statscounter, 2021).

Third, while one might argue that, in the case of Smart Voting, Google search algorithms enabled a less censored selection of information, future cases may be less clear-cut. In particular, specific features of non-free media systems raise concerns about the universality of source prioritization principles. While both Google and Yandex tend to prioritize journalistic media (Zavadski & Toepfl, 2019), our observations demonstrate that their selection of specific outlets is subjected to political bias with the former prioritizing less state-dependent outlets and the latter giving priority to more Kremlin-oriented media. 

It is less clear what Google will do if the number of independent media in Russia continues to decrease, considering the ongoing campaign against journalistic “foreign agents” (Roth, 2021, para 1). Will it shift towards prioritizing the state-sponsored media, following the general policies of the company which are also reflected in its algorithmic systems? Or will these policies be adapted for the case of Russia (and, potentially, other non-free media systems)? To answer these questions, it is important to keep monitoring both how platform algorithms distribute contentious political content and how such distribution is affected by regulation in both Western and non-Western contexts.

Together, our findings provide empirical evidence that may lay ground for a public debate on how and to what degree online intermediaries collaborate with authoritarian governments in enabling censorship. The importance of this debate is amplified by the ongoing transition from traditional violence-based autocracies to informational autocracies. Informational autocracies rely on the regime’s ability to convince the public of its competency (Guriev & Treisman, 2020), which is a task increasingly fulfilled with the help of online intermediaries. Under these circumstances, it is essential to define what behavior is ethically desirable for the corporations working in non-democratic contexts. For instance, should Google collaborate with authoritarian states as it seems to increasingly do in Russia? Or should it non-comply as in the case of Smart Voting and risk fines and, potentially, withdraw from the regional market as in the case of China? 

Answering these questions can be the first step for building up pressure from the public and policymakers to enforce the desirable behavior from the online information intermediaries. Similarly, it can help determine how intermediaries coming from authoritarian states and, potentially, serving as integral components of the informational autocracy shall be treated. Is there a point, for instance, at which Yandex search engine should be viewed as another information asset of the Kremlin, similar to Russia Today, and approached accordingly? 

Findings

Finding 1: Google and Yandex kept displaying results in response to the “Smart Voting” query, despite being ordered to stop doing so.

Despite being ordered to stop displaying search results in response to the “Smart Voting” query, neither Google nor Yandex did so on the eve of the Russian elections. Both search engines kept providing the results throughout the whole period of data collection (September 10–20). This non-compliance can be primarily attributed to the technical implausibility of the initial request. Unlike the requests for Google and Apple to remove the Smart Voting app from their App Stores (Lokot & Wijermars, 2021), which occurred around the same time, the technical realization of non-displaying the results for a particular query is a non-trivial task. While modification of search outputs is not impossible, it usually involves filtering out individual websites from the outputs in general or for specific, but more narrowly defined, queries (e.g., as in the case of the implementation of the right to be forgotten, Mangini et al., 2020). 

The imprecision of the request was also noted by the Yandex representatives, who remarked that “it is unclear what exactly we are requested to do [by the court] and how we can do it” (Interfax, 2021, para 2). Google did not provide any commentary on the decision, but it can be presumed that it also found the court decision difficult to implement. The fact that Yandex stopped prioritizing the links to the Team Navalny resources can be viewed as a form of compliance (albeit more with the legal requirement to stop displaying links to blacklisted websites rather than the interim measure from the trademark case). 

Finding 2: Google and Yandex prioritized different types of sources.

Contrary to earlier studies which did not observe substantive differences in the selection of sources by Yandex and Google (Zavadski & Toepfl, 2017), we discovered substantial discrepancies in the content prioritized by the two engines, with the exception of journalistic media which dominated the outputs in both cases (Figure 3). Google consistently returned links to the Team Navalny resources related to Smart Voting, including the main Smart Voting website that the Russian government had blocked. Yandex did not return any outputs affiliated with the Team Navalny with the exception of September 13, when a link to the Smart Voting website appeared in the results once. We suggest that this exceptional behavior can be attributed to the output randomization used by search engines to maximize user engagement by testing different ways of ranking search results. Sometimes referred to as “Google Dance” (Battelle, 2005), this phenomenon remains largely under-studied (for some exceptions see Haim et al., 2016; Makhortykh et al., 2020), but it can be one of the factors influencing changes in the visibility of individual resources.

We also observed differences in the prevalence of blogs and social media in search outputs. On Yandex, between 20% and 40% of top links were to blogs, while on Google only a few links to blogs appeared during the selected period. For social media the situation was reversed: Google returned more links to the platforms, such as Facebook and YouTube, than Yandex. There also were differences in the affiliation of blogs and social media accounts. Whereas the blogs, in the case of Yandex, usually belonged to anonymous individuals who often promoted pro-Kremlin narratives, the social media pages prioritized by Google usually belonged to Team Navalny. The share of links to reference websites (e.g., Wikipedia) was similar across the two engines.

Finding 3: Content prioritized by Google and Yandex was politically biased.

We found major differences in the share of content with different stances towards Smart Voting (Figure 4). Both search engines returned a similar share of links to neutral content (hovering around 50% of all links), with Google having a higher share of neutral content at the beginning of the data collection period than Yandex. However, the non-neutral content distribution was very different: in the case of Google, almost all non-neutral content had a positive stance towards Smart Voting, while on Yandex the share of pro-Smart Voting content never went above 30% and was consistently lower than the share of anti-Smart Voting content. Thus, we observe a political bias in the outputs of the two search engines.

Among positive and negative assessments of Smart Voting a broad variety of perspectives was found. In the case of anti-Smart Voting content prioritized by Yandex, we observed claims that Smart Voting is a form of manipulation aiming to “deprive the citizens of their right to vote” (Babin, 2020, para 1) or a security threat to personal data (REN TV, 2021a), as well as arguments presenting Smart Voting as ineffective or morally questionable (e.g., because it often recommended voting for the Communist Party) (Novye Izvestiia, 2021). In the case of Google, a positive stance was often expressed in the form of independent media either hosting op-eds that endorsed Smart Voting or linking to the Smart Voting candidate lists.

In some cases, search outputs also included conspiratorial claims related to Smart Voting (Figure 5) such as that it is aiming to harm Russian voters (Lenta, 2021) or is a U.S. Pentagon project designed to interfere in the Russian elections (REN TV, 2021b). Often, these claims originated from Russian officials such as the spokesperson of the Ministry of Foreign Affairs, Maria Zakharova, or the spokesperson of the Russian President, Dmitrii Peskov, that also resulted in them being reiterated by authoritative sources (e.g., established journalistic media). While both Google and Yandex included such claims in their top results, Google tended to prioritize content without mentions of conspiratorial information (e.g., stories about Smart Voting coming from BBC or RBC). By contrast, Yandex gave more visibility to items promoting conspiratorial claims which were usually coming from pro-Kremlin outlets (e.g., TASS or REN TV).

Methods

Data collection

We conducted an agent-based audit of the first page of search results of Yandex and Google for the query “умное голосование” (Smart Voting) (for further details on the chosen auditing approach, which seeks to simulate human browsing behavior, see Appendix, Section A). This specific query was chosen because it is the term to which the court order applies. While users may have also used other search terms, examining the content retrieved for such related (e.g., “SmartVote app”) or derived queries (e.g., “Smart Voting Navalny”) is beyond the scope of the court order and therefore this study.

Data collection started one week before the start of the voting period (September 10) and ended on the last voting day (September 20). To increase the robustness of our results, we aimed to have four rounds of the data collection each day at the same time; however, due to technical issues (e.g., a change in the underlying HTML code of Google’s search page) on September 14 we had only one round of data collection, on September 19 two, and on September 16 three rounds instead of four. Notwithstanding this and a few other limitations (see Appendix, Section B), we suggest that our results are robust enough to infer how information about Smart Voting has been distributed by the two engines.

Data analysis

We extracted all hyperlinks gathered from the organic results on the first page of Yandex and Google search results. Altogether, we collected 731 hyperlinks. The odd number is due to search engines sometimes providing fewer than ten organic results on their first page because of the presence of other elements (e.g., “Knowledge Panels” or “People also ask”). Among the 731 results, there were 119 unique hyperlinks. We manually coded each hyperlinked page into the following categories:

• Whether the page contains information related to Smart Voting: a binary variable (yes/no).
• The type of the page: 1) journalistic media, 2) blog, 3) social media, 4) reference website (e.g., an online encyclopedia), or 5) political campaign (e.g., a Smart Voting-affiliated website). 

Then, for pages containing information related to Smart Voting we used two additional coding categories:

• Whether the page content is politically biased: 1) arguing in favor of Smart Voting, 2) arguing against Smart Voting, 3) neutral regarding Smart Voting.
• Whether the page contains statements about Smart Voting that can be classified as conspiratorial. For example, statements that present Smart Voting as part of secret agenda pursued by domestic (e.g., the Kremlin) or foreign (e.g., the US) actors to interfere in the elections and, potentially, harm the Russian people, were coded as conspiratorial: 1) the page promotes such statements, 2) the page debunks them, 3) the page mentions such statements without adopting clear stance towards them, or 4) the page does not mention any conspiratorial information.

The coding was performed independently by two coders, both fluent in Russian and knowledgeable of the Russian political context (on intercoder reliability, see Appendix, Section C). Finally, we matched the links collected for each of the search engines with their classifications and computed the shares of different types of content distributed by each search engine.