Right and left, partisanship predicts (asymmetric) vulnerability to misinformation

---

Implications

Two years since the call for a systematic “science of fake news” to study the vulnerabilities of individuals, institutions, and society to manipulation by malicious actors (Lazer et al., 2018), the response of the research community has been robust. However, the answers provided by the growing body of studies on (accidental) misinformation and (intentional) disinformation are not simple. They paint a picture in which a complex system of factors interact to give rise to the patterns of spread, exposure, and impact that we observe in the information ecosystem.

Indeed, our findings reveal a correlation between political partisanship, echo chambers, and vulnerability to misinformation. While this confounds the effects of echo chambers and political partisanship on vulnerability to misinformation, the present analysis shows that partisanship is more strongly associated with misinformation. This finding suggests that one approach for social media platforms to combat the spread of misinformation is by prioritizing more diverse, less polarized content. Methods similar to those used here can be readily applied to the identification of such content.

Prior research has highlighted an association between conservative political leaning and misinformation sharing (Grinberg et al., 2019) and exposure (Chen et al., 2020). The proliferation of conspiratorial narratives about COVID-19 (Kearney et al., 2020; Evanega et al., 2020) and voter fraud (Benkler et al., 2020) in the run-up to the 2020 U.S. election is consistent with this association. As social media platforms have moved to more aggressively moderate disinformation around the election, they have come under heavy accusations of political censorship. While we find that liberal partisans are also vulnerable to misinformation, this is not a symmetric relationship: Consistent with previous studies, we also find that the association between partisanship and misinformation is stronger among conservative users. This implies that attempts by platforms to be politically “balanced” in their countermeasures are based on a false equivalence and therefore biased—at least at the current time. Government regulation could play a role in setting guidelines for the non-partisan moderation of political disinformation. 

Further work is necessary to understand the nature of the relationship between partisanship and vulnerability to misinformation. Are the two mutually reinforcing, or does one stem from the other? The answer to this question can inform how social media platforms prioritize interventions such as fact-checking of articles, reining in extremist groups spreading misinformation, and changing their algorithms to provide exposure to more diverse content on news feeds.

The growing literature on misinformation identifies different classes of actors (information producers, consumers, and intermediaries) involved in its spread, with different goals, incentives, capabilities, and biases (Ruths, 2019). Not only are individuals and organizations hard to model, but even if we could explain individual actions, we would not be able to easily predict collective behaviors, such as the impact of a disinformation campaign, due to the large, complex, and dynamic networks of interactions enabled by social media. 

Despite the difficulties in modeling the spread of misinformation, several key findings have emerged. Regarding exposure, when one considers news articles that have been fact-checked, false reports spread more virally than real news (Vosoughi et al., 2018). Despite this, a relatively small portion of voters was exposed to misinformation during the 2016 U.S. presidential election (Grinberg et al., 2019). This conclusion was based on the assumption that all posts by friends are equally likely to be seen. However, since social media platforms rank content based on popularity and personalization (Nikolov et al., 2019), highly-engaging false news would get higher exposure. In fact, algorithmic bias may amplify exposure to low-quality content (Ciampaglia et al., 2018).

Other vulnerabilities to misinformation stem from cognitive biases such as lack of reasoning (Pennycook & Rand, 2019a) and preference for novel content (Vosoughi et al., 2018). Competition for our finite attention also has been shown to play a key role in content virality (Weng et al., 2012). Models suggest that even if social media users prefer to share high-quality content, the system-level correlation between quality and popularity is low when users are unable to process much of the information shared by their friends (Qiu et al., 2017). 

Misinformation spread can also be the result of manipulation. Social bots (Ferrara et al., 2016; Varol et al., 2017) can be designed to target vulnerable communities (Shao et al., 2018b; Yan et al., 2020) and exploit human and algorithmic biases that favor engaging content (Ciampaglia et al., 2018; Avram et al., 2020), leading to an amplification of exposure (Shao et al., 2018a; Lou et al., 2019).

Finally, the polarized and segregated structure of political communication in online social networks (Conover et al., 2011) implies that information spreads efficiently within echo-chambers but not across them (Conover et al., 2012). Users are thus shielded by diverse perspectives, including fact-checking sources (Shao et al., 2018b). Models suggest that homogeneity and polarization are important factors in the spread of misinformation (Del Vicario et al., 2016).

Findings

Finding 1: Users are segregated based on partisanship.

Let us first reproduce findings about the echo-chamber structure of political friend/follower and diffusion networks on Twitter (Conover et al., 2011; 2012) using our dataset. Figure 1 visualizes both networks, showing clear segregation into two dense partisan clusters. Conservative users tend to follow, retweet, and quote other conservatives; the same is true for liberal users. 

Finding 2: Vulnerability to misinformation, partisanship, and echo chambers are related phenomena.

We want to test the hypothesis that partisanship, echo chambers, and the vulnerability to misinformation are related phenomena. We define a Twitter user’s vulnerability to misinformation by the fraction of their shared links that are from a list of low-quality sources. Consistent with past research (Grinberg et al., 2019), we observe that vulnerability to misinformation is strongly correlated with partisanship in right-leaning users. However, unlike past work, we find a similar effect for left-leaning users (Figure 2a). Overall, we observe that right-leaning users are slightly more likely to be partisan and to be vulnerable to misinformation. The great majority of users on both sides of the political spectrum have misinformation scores below 0.5, indicating that those who share misinformation also share a lot of other types of content. In addition, we observe a moderate relationship between vulnerability to misinformation and two measures that capture the extent to which a user is in an echo chamber: the similarity among sources of links shared by the user and their friends (Figure 2b) and the clustering in the user’s follower network (Figure 2c).

In Figure 3 we show that the three independent variables we analyze (partisanship, similarity, and clustering) are actually correlated with each other. User similarity and clustering are moderately correlated, as we might expect given that they both aim to capture the notion of embeddedness within an online echo chamber. Partisanship is moderately correlated to both echo chamber quantities.

Finding 3: Vulnerability to misinformation is most strongly influenced by partisanship.

Given these correlations, we wish to disentangle the effect of partisanship and echo chambers on vulnerability to misinformation. To this end, we conducted a regression analysis using vulnerability to misinformation as the dependent variable. The results are summarized in Table 1. We see that all independent variables except clustering for left-leaning users significantly affect the dependent variable (p < 0.01). However, as shown by the coefficients and the adjusted increases in R², the effect is much stronger for partisanship.

Identifying the most vulnerable populations may help gauge the impact of misinformation on election outcomes. Grinberg et al. (2019) identified older individuals and conservatives as particularly likely to engage with fake news content, compared with both centrists and left-leaning users. These characteristics correlate with those of voters who decided the presidential election in 2016 (Pew Research Center, 2018), leaving open the possibility that misinformation may have affected the election results.

Our analysis of the correlation between misinformation sharing, political partisanship, and echo chambers paints a more nuanced picture. While right-leaning users are indeed the most likely to share misinformation, left-leaning users are also significantly more vulnerable than moderates. In comparing these findings, one must keep in mind that the populations are different—voters in the study by Grinberg et al. (2019) versus Twitter users in the present analysis. Causal links between political bias, motivated reasoning, exposure, sharing, and voting are yet to be fully explored. To confirm that partisanship is associated with vulnerability to misinformation for both left- and right-leaning users, we conducted regression analysis comparing partisan (liberal or conservative) against moderate users. To this end, we grouped all users together and took the absolute value of their partisanship score as an independent variable. The results from this analysis (Table 2) show that partisan users are more vulnerable than moderate users, irrespective of their political leanings.

Methods

Dataset

We collected all tweets containing links (URLs) from a 10% random sample of public posts between June 1 and June 30, 2017, through the Twitter API. This period was characterized by intense political polarization and misinformation around the world and especially in the U.S. (Boxell at al., 2020). Major news events included the announcement of the U.S. withdrawal from the Paris Agreement; reinstatement of the ban on immigration ban from Muslim- majority countries; emerging details from the Muller investigation about Russian interference in the 2016 U.S. election; American and Russian interventions in Syria; James Comey’s Senate Intelligence Committee testimony; the Congressional baseball shooting; U.S. restrictions on travel and business with Cuba; a special congressional election in Georgia; and continuing efforts to repeal Obamacare. 

Since we are interested in studying the population of active online news consumers, we selected all accounts that shared at least ten links from a set of news sources with known political valence (Bakshy et al., 2015; see “Partisanship” section below) during that period. To focus on users who are vulnerable to misinformation, we further selected those who shared at least one link from a source labeled as low-quality. This second condition excludes 5% of active right-leaning users and 30% of active left-leaning users. Finally, to ensure that we are analyzing human users and not social bots, we used the BotometerLite classifier (Yang et al., 2020) to remove likely bot accounts. This resulted in the removal of a little less than 1% of the accounts in the network. The user selection process resulted in the dataset analyzed in this paper, whose statistics are summarized in Table 3.

Network analysis

We created two networks. First, we used the Twitter Friends API to build the follower network among the users in our dataset. Second, we considered the retweets and quotes in the dataset to construct a diffusion network. Both networks are very dense and, consistently with previous findings, have a deeply hierarchical core–periphery structure (Conover et al., 2012). Application of k-core decomposition (Alvarez-Hamelin et al., 2005) to the follower network reveals a core with k = 940, meaning that each user has at least 940 friends and/or followers among other accounts within the network core. Since the network includes more than 4 million edges, in Figure 1a we sample 10% (432,745) of these edges and then visualize the k = 2 core of the largest connected component. The retweet/quote network has a k = 21 core and includes 111,366 edges. In Figure 1b we visualize the k = 2 core of the largest connected component. 

Partisanship

To define partisanship, we track the sharing of links from web domains (e.g., cnn.com) associated with news sources of known political valence. For a source of political valence, we use a dataset compiled by Facebook (Bakshy et al., 2015), which consists of 500 news organizations. By examining the Facebook page of each news organization, the political valence score was computed based on the political self-identification of the users who liked the page. The valence ranges from –1, indicating a left-leaning audience, to +1, indicating a right-leaning audience.

We define the partisanship of each user  as

where is the fraction of links from source  that  shares, derived from the Twitter data, and  is the political valence of source . In correlation and regression calculations, we take the absolute value  for left-leaning users.

This definition of partisanship implicitly assumes that the sharing of a link implies approval of its content. While this is not necessarily true all of the time, we can reasonably assume it is true most of the time. To verify this, we sampled a subset of the accounts in our dataset (246 left- and 195 right-leaning users) and examined their profiles. Figure 4 visualizes the most common hashtags in the profile descriptions, confirming that our partisanship metric closely matches our intuitive understanding of left- and right-leaning users.

Misinformation

To define misinformation, we focus on the sources of shared links to circumvent the challenge of assessing the accuracy of individual news articles (Lazer et al., 2018). Annotating content credibility at the domain (website) level rather than the link level is an established approach in the literature (Shao et al., 2018b; Grinberg et al., 2019; Guess at al., 2019; Pennycook & Rand, 2019b; Bovet & Makse, 2019). 

We considered a list of low-quality sources labeled by human judges (Zimdars et al., 2017; Zimdars, 2016). While the process employed for annotating sources is well-documented, the list is no longer maitained and its curation may be subject to the bias of the judges. Nevertheless, the list was current when our data was collected, and therefore has good coverage of low-credibility sources that were active at that time. More current lists of low-credibility domains do not allow for equal coverage because many sources have since become inactive. The annotation categories used to identify low-credibility sources in our dataset are shown in Table 4, along with statistics about their prevalence, as well as the numbers of links to news and other sources. Note that the total number of links shared is greater than the total number of tweets, for two reasons. First, a user can share multiple links in a single tweet; and second, most misinformation sources have multiple labels applied to them. 

We extracted all links shared by each user, irrespective of whether they were from legitimate news, misinformation, or any other source. Each user’s  misinformation score  is the fraction of all links they shared that were from sources labeled as misinformation. 

User similarity

To measure similarity between users, we constructed the matrix of TF-IDF values for the user-domain matrix, with users analogous to documents (rows), and domains analogous to terms (columns). Thus, each user is represented by a vector of TF-IDF values indicating how strongly they are associated to each domain:

where  is a user, is a domain,  is the frequency with which  shares from , and 

with  being the subset of users who have shared from domain , out of the set of users .

User clustering

To capture user clustering based on the follower network, we computed the clustering coefficient for each user as:

where is the number of directed triangles through the user node , is the sum of in-degree and out-degree for , and is the reciprocal degree of (Fagiolo, 2007). This measure quantifies how densely interconnected a user’s social network is. A dense follower network is another way to quantify an echo chamber.

Regression analysis

To compare the different factors that may contribute to a user’s misinformation score, we used multiple linear regression. Since the regression was performed for right- and left-leaning users separately, we took the absolute values of the partisanship scores. In addition, we took the z-score transform of each variable. The resulting regression equation is: