“Fake news” may have limited effects beyond increasing beliefs in false claims

---

Implications

The combination of observational and experimental approaches that we employ offers important new insights into the effects of “fake news.” Most notably, both types of data indicate how untrustworthy online content can promote inaccurate factual beliefs. Using our real-world consumption data, we see that people who visit untrustworthy websites are more likely to hold a misperception.  As a result, we are confident that exposure to misinformation increases misperceptions about specific and relevant factual claims.

By contrast, our findings often conflict when we examine the relationship with other outcomes of interest. For instance, our regression analysis reveals that consuming information from untrustworthy websites is associated with lower media trust, but reading a false article in our experiment did not worsen attitudes towards the media. This result suggests that people may choose to consume factually dubious content because they lack trust in the media. We observe a similar discrepancy between our observational and experimental data for political participation. Respondents who visit untrustworthy websites report similar levels of political participation to those who do not visit such websites. Experiments can be a powerful tool to help show causal effects that are unclear from observational data alone. Randomized exposure to a single false article increased respondents’ self-reported likelihood to vote. This experimental effect is likely short-lived, but underscores the broader point that further research is needed to understand whether and how false content affects political attitudes and behavior other than factual beliefs. 

To make further progress, future studies of the effects of “fake news” must confront three additional challenges. First, our measures of untrustworthy website consumption do not capture exposure on social media platforms or mobile devices. As a result, we cannot measure exposure to headlines that people see in a news feed but do not click or website visits that take place on devices such as phones and tablets (the behavioral data we study only includes limited data from mobile devices). Second, it is unclear how much untrustworthy news content should be shown to participants in studies. The effects of exposure to this type of content may only be measurable if we administer a higher dosage than our current studies provide, which in turn raises ethical concerns that researchers should consider carefully. Finally, we should test whether the effects we find in our experiments would be observed among people who would consume untrustworthy websites in the real world. It is possible that Americans who visit these sites do not react as strongly to them as others would.

Until these challenges are addressed, it is difficult to fully assess the effects of “fake news.” Our results suggest, however, that this type of content has limited effects on political attitudes and behavior beyond factual beliefs. The most worrisome aspect of these untrustworthy websites, in other words, is that they often succeed in misleading Americans.

Findings

Each finding below is illustrated using one or more graphs reporting the average difference in the outcome measure between respondents who did not consume information from untrustworthy websites and those who did (for the non-experimental data) and/or the causal effect of exposure to a pro-Democratic or pro-Republican false story on the outcome measure (for the experimental data). We report differences separately for Democrats and Republicans in some cases. 

Finding 1: Consumption of untrustworthy websites is associated with more negative feelings toward the media. 

We find that Americans who consume information from untrustworthy websites tend to report less “trust and confidence” in the media and more negative feelings towards it (see Figures 1a and 1b). Previous research has shown mixed evidence about the connection between views toward mainstream news and untrustworthy website consumption. Low media trust can be associated with alternative media consumption (Tsfati and Cappella 2003), though more recent research has called for a re-examination of this relationship (Kalogeropoulos et al. 2019). Similarly, behavioral evidence suggests that consumption of news from untrustworthy websites does not crowd out consumption of mainstream news, which is consistent with evidence that many people who view the media negatively still consume a great deal of mainstream news (Guess et al. 2019a,b). 

Finding 2: Consumption of untrustworthy websites is associated with polarized feelings toward the political parties. 

We find evidence that the consumption of untrustworthy websites is associated with more polarized feelings towards the two political parties (see Figure 1b). Previous research examining other forms of media content suggests that exposure to pro-attitudinal information and partisan criticism (the types of content that untrustworthy websites often provide) can inspire more polarized attitudes toward the parties (Iyengar et al. 2012; Garrett et al. 2014; Suhay et al. 2018). We assess whether the relationship holds for this relatively new type of digital content. In our fall and winter 2018 data, which were collected immediately before and after the midterm elections, we find greater affective polarization (which we measure as the difference in people’s feelings toward their preferred party and the opposition party) among consumers of information from untrustworthy websites.

Finding 3: Untrustworthy website consumption is associated with greater beliefs in pro-GOP misperceptions and reduced belief in pro-Democratic ones.

People who consume information from untrustworthy websites report greater belief in pro-Republican misperceptions and less in pro-Democratic misperceptions even after we account for partisanship, ideology, and feelings toward President Trump (see Figure 2). Previous research found that the untrustworthy websites which attracted the most attention and traffic before the 2016 election had a conservative or pro-Republican orientation (e.g., Benkler et al. 2018; Grinberg et al. 2019; Guess et al. 2019b). We infer that this pattern of slant explains the positive (negative) relationship between visiting untrustworthy websites and pro-Republican (pro-Democratic) misperceptions, though we cannot determine whether this result reflects self-selection or exposure to misinformation from these sites.

Finding 4: The consumption of untrustworthy websites is generally not associated with political participation intentions.

Our data indicate that Americans who consume news from untrustworthy websites generally had similar intentions to participate in politics to those who do not visit such websites (see Figure 2). Given that anger helps to mobilize voters (e.g., Valentino et al. 2011) and that articles on untrustworthy websites are often irate and inflammatory, a frequent question is whether this content increases intent to engage in political activities. However, visiting untrustworthy websites was not associated with intent to vote in the next election. In fall 2018, respondents who visited untrustworthy websites reported significantly greater intent to engage in non-voting forms of political action, but this relationship was not observed in summer 2018. Subsequent research should examine if the pattern holds in future elections.

Finding 5: Randomized exposure to a dubious article increases belief in the article’s central claim, especially for Democrats when the claim was congenial.

Respondents who were randomly exposed to an inflammatory but misleading article from an untrustworthy website rated the article’s central claim as more accurate than those who did not read the article in question (see Figure 3). This effect was magnified for Democrats exposed to a pro-Democratic false story. We ran this experiment as part of the fall 2018 survey, which randomly assigned respondents to one of three articles: a politically neutral story on food, a false story about how George Soros was sponsoring a migrant caravan from Central America (a prominent conspiracy theory at the time), and a false story suggesting that Jared Kushner, Donald Trump’s son-in-law, played a role in the death of Jamal Khashoggi at the hands of Saudi Arabian intelligence agents (a more obscure conspiracy theory). We found that exposure to these stories, which are provided in Online Appendix A, increased respondents’ beliefs in the underlying claims. However, the effect was not symmetric by party. Exposure to the Kushner/Kashoggi article increased belief in its central claim among identifiers with both major parties, but the effect was differentially larger among Democrats compared to Republicans. By contrast, the increases in belief in the Soros/caravan conspiracy claim resulting from exposure to the article in question were similar between parties. These results may reflect differences in pre-treatment awareness of these stories since the Soros/caravan rumor was covered more widely in the period before the study was fielded; prior encounters with information can reduce or change its impact (Druckman and Leeper 2012). ¹We omit the effect of exposure to one false article on respondent belief in the claim expressed in the other false article from the figure. In one of four cases, exposure to a false article decreased belief in the other claim (ratings of the Soros/caravan claim among Republicans who saw the Kushner/Khashoggi article; see Table B5). Given that we only observe one effect of this type, we are not certain if these results are meaningful. 

Finding 6: Randomized exposure to a dubious article increases intent to vote. However, article exposure has no measurable effect on other forms of participation, views toward the media, or on affective polarization.

The effects of exposure to dubious articles are largely limited to belief in the false claims they promote. Among all the other outcome measures we tested, the only variable affected by exposure to a dubious article is vote intention (see Figure 4). People who receive a false pro-Democratic or pro-Republican article reported a greater likelihood of voting in the upcoming election. Importantly, these effects did not differ significantly by congeniality of article content (see Tables B8–B10 in Online Appendix B). Respondents were more likely to say they would vote regardless of whether the article favored their preferred party; exposure to the article did not differentially mobilize supporting partisans or demobilize opponents. Moreover, these effects did not extend to other important attitudes and behavioral intentions. Specifically, exposure to a false article did not make respondents feel more negatively toward the media or the other party — we find no measurable effect on attitudes toward the media or affective polarization.

Methods

To test our hypotheses, we draw on data from three two-wave survey panels conducted by the survey company YouGov during and immediately following the 2018 U.S. midterm elections: 

• A two-wave panel study fielded June 25–July 3, 2018 (wave 1; N = 1,718) and July 9–17, 2018 (wave 2; N = 1,499), which we refer to as the summer 2018 data; 
• A two-wave panel study fielded October 19–26 (wave1; N=3,378) and October 30–November 6, 2018 (wave 2; N = 2,948), which we refer to as the fall 2018 data; 
• A two-wave panel study fielded November 20–December 27, 2018 (wave 1; N = 4,907) and December 14, 2018–January 3, 2019 (wave 2; N = 4,283), which we refer to as the winter 2018 data. 

Respondents were selected by YouGov’s matching and weighting algorithm to approximate the demographic and political attributes of the U.S. population. Participants were ineligible to take part in more than one study. 

These surveys were designed to include a separate experiment evaluating whether a media literacy intervention affects how people evaluate both mainstream and false news headlines. The results of this experiment are examined in a separate study. Due to a programming error, all respondents in the summer and fall surveys were exposed to the intervention in the first wave of the two-wave survey. The experiment reported in this article was then conducted in the second wave in each of the summer and fall surveys. Given that all respondents also received the media literacy intervention, the effect sizes of our false and inflammatory news exposure treatments may underestimate the true effects of these articles. Future studies that we intend to publish will include additional analyses that were preregistered prior to conducting each of these studies. 

Information consumption in this study is measured using behavioral data on respondents’ web visits collected unobtrusively in the 1-3 week period before the initial survey wave. Data is available from users’ laptop or desktop computers as well as their mobile devices and tablets (though coverage of the latter is limited). ²Web visits are collected anonymously with users’ permission through a mix of browser plug-ins, proxies, and VPNs. The providers of this data were the firms Wakoopa (summer 2018) and Reality Mine (fall and winter 2018). For each survey, we created a binary measure indicating whether we observe a visit to an untrustworthy website in the 1–3 week period for which we have data prior to the respondent completing the survey. This approach allows us to measure consumption of dubious online information directly rather using survey questions and hoping that respondents both remember and faithfully report all the websites they visited. 

We use regression to first examine associations between prior consumption of news from un- trustworthy websites and feelings toward the media and the political parties, belief in topical mis- perceptions, and intent to vote or engage in other forms of political participation. Misperception questions focused on topics such as Trump’s immigration policy, the Special Counsel investigation of Russian interference in the 2016 United States elections, and Brett Kavanaugh’s Supreme Court nomination. These findings are correlational and cannot establish causality. In addition, because intention to vote or engage in other forms of political misinformation were only measured after the experiment, we consider only individuals assigned to the control condition in observational analyses of these outcome measures. 

We also use experiments to assess the effects of exposure to a false and inflammatory article from an untrustworthy website. In the second wave of the summer and fall surveys, respondents were randomly assigned to read a false and inflammatory pro-Democratic article with 1/3 probability, a false and inflammatory pro-Republican article with 1/3 probability, or an unrelated placebo article with 1/3 probability. (These treatments and relevant question wordings are provided in Online Appendix A.) This approach allows us to measure the causal effect of exposure to an article from an untrustworthy website with greater confidence than we can using non-experimental data. 

Because we frequently expect different results by party, we report results for Democrats and Republicans (including those who lean toward one party) only; independents who do not lean toward other party are excluded. This exclusion simplifies interpretation of our results.