Peer Reviewed

Research note: Does the public support fact-checking social media? It depends who and how you ask

We analyze original survey data on support for social media companies’ fact-checking of politicians in general and President Trump in particular. We find overwhelming majorities of Democrats support fact-checking in both instances, while a majority of Republicans support fact-checking of politicians in general but not of President Trump. Moreover, we find those concerned about contracting COVID-19 are more likely to support fact-checks. Rather than viewing fact-checks as a cure-all, we argue the importance of acknowledging their limitations and identifying when and on whom efforts may be effective.

Image by Gerd Altmann on Pixabay

Research Questions

  • Does the public support social media fact-checking of politicians? 
  • Does the public differentiate fact-checks of politicians in general versus President Trump specifically?
  • Does concern about COVID-19 influence support for fact-checking?

Essay Summary

  • We conducted an experimental web survey via mTurk, an Amazon crowdsourcing platform, to identify support for fact-checking politicians and President Trump separately. We then employed statistical analysis to identify differences by party identification and other demographic factors. 
  • We find that majorities supported fact-checking with clear differences among Democrats and Republicans. Democratic support shows marginal differences between fact-checks of politicians versus President Trump. A majority of Republicans support fact-checks of politicians but not of Trump.
  • Concern about contracting COVID-19 corresponds with support for fact-checks with a larger effect among Republicans.
  • Fact-checking interventions work only for those people who see value in fact-checking.

Implications

Donald Trump has notably made thousands of factually inaccurate statements on Twitter. In May of 2020, for the first time, Twitter fact-checked a Trump tweet on mail-in voting (Mangan & Brueninger 2020). Twitter also recently put labels on a selection of Trump’s tweets that violated company policies (Feiner 2020). Facebook, originally reluctant to act, announced in June that it would add labels to posts on voting directing users to credible election information (Mihalcik 2020). These actions sparked new questions about the role of social media companies in fact-checking statements by politicians. Such debates have increased importance due to the COVID-19 pandemic as medical misinformation can result in prolonging the pandemic and increasing the number of deaths.

DateEventSource
December 31st, 2019First COVID-19 case reported in China(WHO)
April 28th, 2020YouTube begins fact-checking videos in the United States(YouTube)
Early May, 2020Social media platforms struggle to deal with “Plandemic” misinformation video (CNBC)
May 27th, 2020Twitter fact-checks Trump post for the first time(CNN)
May 28th, 2020Mark Zuckerberg says Social media shouldn’t fact check political speech(CNBC)
June 22nd, 2020Google Images begins to fact-check image results(Google)
July 21st, 2020Facebook begins labeling posts from Trump and Biden(CNN)
August 7th, 2020Facebook memos show misinformation policies were not enforced against conservative repeat offenders(NBC)
September 4th, 2020Facebook refuses to remove a Trump post encouraging people to vote twice(BuzzFeed)
October 6th, 2020Facebook takes down Trump post comparing COVID-19 to the flu(BuzzFeed)
Table 1. Summary of Key Fact-Checking Events During the COVID-19 Crisis.

Social media fact-checks can take multiple forms, from Facebook collaborating with a third-party organization to label misinformation and post fact-checks under such posts (Bode & Vraga, 2015) to Twitter’s internal methods to categorize posts as misleading, disputed, or unverified (Roth & Pickles, 2020). Who should perform fact-checks and in what capacity further shapes perceptions, and perhaps perceived biases, of the process? Table 2 provides a snapshot of current fact-checking policies and procedures among four major social media platforms.

PlatformPoliciesSource
FacebookFacebook identifies misinformation after it goes viral and refers it to third party fact-checkers before applying a label. Punishments are threatened for repeat offenders. (Facebook)
TwitterNo comprehensive misinformation policies exist. Twitter will label or remove manipulated media, or anything intended to misinform or interfere with elections or other civic processes. Exceptions are made for key elected officials.(Twitter)
YouTubeYouTube provides information from third-party fact-checkers on specific searches. Specific videos are not fact-checked.(Google)
PinterestPinterest says they will remove content from conspiracy theories, medical misinformation, election misinformation, and disinformation campaigns. Manipulated media will also be removed. The fact-checking process is not outlined.(Pinterest)
Table 2. Summary of Social Media Fact-Checking Policies.

Experimental research commonly finds a positive relationship between support for fact-checking and political knowledge (e.g. Thorson 2016; Fridkin et al. 2015). However, research rarely addresses that partisans seldom seek contrasting views (e.g. Iyengar & Hahn 2009) and may view fact-checks differently (e.g. Glynn & Huge 2014; Nyhan & Reifler 2015; Walker & Gottfried 2019; Robertson et al. 2020; Lyons et al. 2020). Shin and Thorson (2017) find not only selective sharing of fact-checks based on partisan preferences, but hostility towards outgroup fact-checks, especially among Republicans. This may be due to analyses of fact-checks labeling Republican politicians’ claims false at a higher rate than Democratic counterparts (e.g. Center for Media and Public Affairs at George Mason University 2013; Ostermeier 2011). Thus, fact-checks may only influence those that already see value and credibility in the fact-checking process. 

Despite the positive normative implications of fact-checking, we expect differences of opinion on fact-checking based on partisanship and whether politicians or Trump specifically were targets of fact-checks. In addition, the role of misinformation regarding COVID-19 (e.g. Krause et al. 2020) may increase support for fact-checks. For example, President Trump claimed that 99% of COVID-19 cases were harmless (Dugyala & Forgey 2020) and made 654 false statements (Dale & Subramanian 2020) about the pandemic during the first 14 weeks of the pandemic by CNN’s count. 

Despite noble intentions, there is little evidence that fact-checks change support for candidates (e.g. Nyhan et al. 2019; Barrera et al. 2020), and instead could lead Republicans to disregard or dismiss efforts as partisan attacks. This could produce more partisan rancor and cause fact-checking to be counterproductive in attempting to improve general political discourse. Our evidence suggests that for some people, fact-check efforts of politicians may never work. However, particular concerns about health and safety, such as contracting COVID-19, may increase interest in social media fact-checks. However, fact-checks alone, even if limited to public health issues, likely are not enough to correct misinformation if the sources and impartiality of the fact-checks themselves are in dispute.

Findings 

This study investigates fact-checks conducted internally by social media platforms and support for these efforts based on partisanship, the target of fact-check efforts, and concern for contracting COVID-19. This includes three important findings on the extent of support for fact-check efforts.

Finding 1: Democrats are more supportive of fact-checks overall, while support among Republicans drops when the focus is on Donald Trump.

Figure 1 shows clear differences between how Democratic and Republican respondents evaluate the two versions. For visual clarity, we recoded the five-point scale into three categories by combining “strongly disagree” with “disagree” and “strongly agree” with “agree”.

The results show Democrats as more supportive of fact-checks overall than Republicans. More interesting is that Republican support for fact-checking Trump dropped by 15 percentage points in comparison to fact-checking politicians in general, with a difference in means test statistically significant at the .01 level. One potential reason for the decline in support is that Republicans may interpret the prompt as unfairly singling out Trump, while supporting fact-checks of politicians of politicians that they do not like.  

Similarly, because Republicans tend to exhibit greater positive affect for their party than Democrats, the perception of Republicans being fact-checked may produce defensive reactions. Trump’s status as an unconventional politician may also be motivating such variation among Republicans. Surprisingly, Democrats’ support for fact-checking did not increase when specifying President Trump in the statement. This may be because Trump was already on the minds of those presented with the “politicians” version of the statement since he tends to be fact-checked by the media daily (Kessler et al. 2020).

Figure 1. Support for fact-checks of politicians and President Trump. 

Finding 2: Those concerned about contracting COVID-19 statistically were more likely to support fact-checks.

We found that people who were concerned about contracting COVID-19 statistically were more likely to support fact-checks, regardless of which fact-check statement they received. Overall, 65.04% of Democrats were worried about contraction (a four or five on a five-point scale), compared to only 42.7% of Republicans. Figure 2 below separates respondents by their concern for contracting COVID-19 and support for fact-checks under each version. Overall, we see support for fact-checks of politicians and Trump in particular are roughly 36% higher among those concerned about COVID-19. Among Democrats concerned about COVID-19, we see a larger increase for fact-checking politicians (24.66%) compared to Trump (15.17%), while among Republicans we see the opposite, with COVID-19 concerns increasing support for fact-checks of Trump (42.84%) compared to politicians more broadly (31.11%).

Figure 2. Support for fact-checks based on level of concern on contracting COVID-19.

Finding 3: The effects of the framing of fact-checks and concern about contracting COVID-19 remain after considering demographic factors and ideological extremity. 

We ran separate ordered logistic regression models on support for social media fact-checks for Democrats and Republicans. The models included whether respondents received the Trump version of the fact-check statement, along with basic demographic factors (gender, age, education, and household income), ideological extremity, and concern about contracting COVID-19.

For Democrats, those receiving the Trump version corresponded with greater support for fact-checks (.35 on the five-point scale), significant at p = .05, although with wide confidence intervals. Age also showed a small positive influence (.03) on fact-check support (p < .001), with ideological extremity having the largest positive influence (.58, p <.001). Lastly, concern over COVID-19 corresponds with .32 point increase in support for fact-checks, significant at p = .001. 

Predicted probabilities find that Democrats that received the Trump version were more likely to support fact-checks (90% vs. 86%). In addition, those most worried about contracting COVID-19 (a five on the five-point scale) were 14-18% more supportive of fact-checks than the least worried (politicians version: 90% vs. 72%; Trump version 93% vs. 79%).  

For Republicans, several factors were statistically significant, most notably that receiving the Trump version corresponded with the largest decline in support (-.75 on a five-point scale, p =.001), again with wide confidence intervals. Women showed the largest positive correlation with fact-checking (.58, p = .008), while age had a minor substantive impact (-.16, p =.009). Unlike Democrats, ideological extremity was not statistically significant among Republicans. Finally, and similar to Democrats, concerns about contracting COVID-19 corresponded with a support for fact-checks (.42, p <.0001). 

Predicted probabilities for fact-checks show that Republicans receiving the Trump version were less supportive of fact-checks by 18% (politicians version: 58%, Trump version: 39%). Like Democrats, those most worried about contracting COVID-19 were more supportive of fact-checks than the least worried, however here the variation is 38-39% (politicians version: 75% vs. 36%; Trump version 59% vs. 21%). Put another way, while Republican support for fact-checks lagged behind that of Democrats, we find that concern for contracting COVID-19 actually exhibits a larger substantive effect on support for fact-checks among Republicans. 

Figure 3. Coefficients of support for social media fact-checks among Democrats and Republicans. 

Methods 

We asked our study participants three questions: does the public support social media fact-checking, does public support differ when focused on politicians in general versus President Trump specifically, and does concern about contracting COVID-19 influence support for fact-checking? 

To test our questions and isolate the views of fact-checks on politicians versus President Trump, we used an original web survey as we could randomly assign respondents differing versions of our fact-check question. We relied on an original web survey, recruiting 1,027 American respondents 18-79 years of age via mTurk on July 7, 2020. Participants were on average 40 years old, 55% of which were male, and largely educated, with 65% having at least a Bachelor’s degree. This method of recruitment allowed for a regionally, ideologically, and ethnically diverse sample. mTurk recruitment is also commonplace in experimental research in social science. 

We asked respondents first a series of demographic questions, across several pages, and respondents could not change their answers after leaving a page. We then randomly assigned respondents to one of two statements and asked them to rank their support for fact-checking either Trump or politicians on a five-point Likert scale (“strongly disagree” to “strongly agree”).

  • Version 1: Social media companies should provide fact checks of statements from politicians.
  • Version 2: Social media companies should provide fact checks of statements from President Trump.

Besides our experimental question addressed earlier, we ask respondents to evaluate the following statement on a 1 to 5 scale: “I am concerned about contracting COVID-19 (coronavirus).” This question allows us to look for potential relationships between fear of COVID-19 and support for fact-checks on social media.

We followed descriptive data with ordered logit regression analysis in order to identify the size of the effect of framing fact-checks in a generic fashion versus on a particular politician, controlling for potentially important demographic factors. Ordered logit models, rather than ordinary least squares (OLS) regressions are justified when the dependent variable is a Likert scale. Running separate models by partisan identification, we included controls for gender, age, household income, education, ideological extremity (how far away from a moderate position of a four on a seven-point ideology scale), and concern about contracting COVID-19. We then generated predicted probabilities for support for fact-checking (a four or five on our five-point scale) at different levels of concern for COVID-19 and for the fact-check prompt received, holding all other independent variables at their mean. This allows us to identify the substantive effect of COVID-19 concerns on support for fact-checking, ultimately showing a twenty percent or greater increase in the likelihood of support for fact-checks as one moves from not concerned to concerned about contraction. 

Limitations and robustness

We use an experimental design embedded in a web survey to identify how partisanship and concern about COVID-19 influence views of fact-checking; however, several caveats are necessary. First, our recruitment via mTurk is a convenience sample, and while appropriate for experimental randomizations, does not perfectly mirror the national population. This may affect generalizability. Secondly, our research design only tests one specific politician, Donald Trump, and not others. For example, we cannot determine whether Democrats would be less supportive of fact-checks or Republicans more supportive if the politician mentioned was a Democrat (e.g. Joe Biden). Future work should include such tests, as well as testing whether the public elicits greater support of fact-checks based on the gender, experience, or level of office of the politician. 

We did not provide additional information about social media fact-checks, so it is possible that partisans interpreted the basic question differently and counter to the intention of the experiment. Nor can we fully parse out pre-existing and often partisan beliefs on the source of misinformation (e.g. Stocking, Grieco, & Gottfried, 2019) or reliable media outlets (e.g. Nielsen & Graves, 2017). Partisans may also have divergent baseline views of both social media and fact-checks in general, which admittedly cannot be disaggregated in this research design. Lastly, we cannot parse out support for different forms of fact-checks (e.g. internal or third-party efforts).

Topics
Download PDF
Cite this Essay

Rich, T.S., Milden, I., Wagner, M.T. (2020). Research note: Does the public support fact-checking social media? It depends who and how you ask. The Harvard Kennedy School (HKS) Misinformation Review. https://doi.org/10.37016/mr-2020-46

Bibliography

Barrera, O., Guriev, S., Henry, E., & Zhuravskaya, E. (2020). Facts, alternative facts, and fact checking in times of post-truth politics. Journal of Public Economics182https://doi.org/10.1016/j.jpubeco.2019.104123

Bode, L., & Vraga, E. K. (2015). In related news, that was wrong: The correction of misinformation through related stories functionality in social media. Journal of Communication, 65(4), 619-638. https://doi.org/10.1111/jcom.12166

The Center for Media and Public Affairs at George Mason University. (2013, October 30). Study: Media

fact checker says Republicans lie more. The Center for Media and Public Afairs. https://cmpa.gmu.edu/study-media-fact checker-says-republicans-lie-more/

Dale, D., & Subramaniam, T. (2020, May 29). Fact check: Breaking down Trump’s 654 false claims during the coronavirus pandemic. CNNhttps://www.cnn.com/2020/05/29/politics/fact check-trump-coronavirus-pandemic-dishonesty/index.html

Dugyala, R., & Forgey, Q. (2020, July 5). Trump’s ‘99 percent’ coronavirus comment finds little support. Politicohttps://www.politico.com/news/2020/07/05/hahn-coronavirus-trump-infection-348954

Feiner, L. (2020, June 23). Twitter flagged another Trump tweet for violating its policies. CNBChttps://www.cnbc.com/2020/06/23/twitter-labeled-another-trump-tweet-for-violating-its-policies.html

Fridkin, K., Kenney, P. J., & Wintersieck, A. (2015). Liar, liar, pants on fire: How fact checking influences citizens’ reactions to negative advertising. Political Communication32(1), 127-151. https://doi.org/10.1080/10584609.2014.914613

Glynn, C. J., & Huge, M. E. (2014). How pervasive are perceptions of bias? Exploring judgments of media bias in financial news. International Journal of Public Opinion Research26(4), 543-553. https://doi.org/10.1093/ijpor/edu004

Iyengar, S., & Hahn, K. S. (2009). Red media, blue media: Evidence of ideological selectivity in media use. Journal of Communication59(1), 19-39. https://doi.org/10.1111/j.1460-2466.2008.01402.x

Kessler, G., Rizzo, S., & Kelly, M. (2020, July 13). President Trump has made more than 20,000 false or misleading claims. The Washington Posthttps://www.washingtonpost.com/politics/2020/07/13/president-trump-has-made-more-than-20000-false-or-misleading-claims/

Krause, N. M., Freiling, I., Beets, B., & Brossard, D. (2020). Fact checking as risk communication: The multi-layered risk of misinformation in times of COVID-19. Journal of Risk Research, 1-8. https://doi.org/10.1080/13669877.2020.1756385

Lyons, B., Mérola, V., Reifler, J., & Stoeckel, F. (2020). How politics shape views toward fact checking: Evidence from six European countries. The International Journal of Press/Politics25(3), 469-492. https://doi.org/10.1177/1940161220921732

Mangan, D., & Breuninger, K. (2020, May 26). Twitter fact checks Trump, slaps warning labels on his tweets about mail-in ballots. CNBChttps://www.cnbc.com/2020/05/26/twitter-fact checks-trump-slaps-warning-labels-on-his-tweets-about-mail-in-ballots.html

Mihalcik, C. (2020). Facebook to label posts about voting from political candidates. CNET. https://www.cnet.com/news/facebook-to-label-posts-about-voting-from-political-candidates/

Nielsen, R., K., & Graves., L. (October 2017). “News you don’t believe”: Audience perspectives on fake news. Reuters Institute. https://reutersinstitute.politics.ox.ac.uk/our-research/news-you-dont-believe-audience-perspectives-fake-news

Nyhan, B., Porter, E., Reifler, J., & Wood, T. J. (2019). Taking fact checks literally but not seriously? The effects of journalistic fact checking on factual beliefs and candidate Favorability. Political Behaviorhttps://doi.org/10.1007/s11109-019-09528-x

Nyhan, B., & Reifler, J. (2015). Estimating fact checking’s effects: Evidence from a long term experiment during campaign 2014. American Press Institute. https://www.americanpressinstitute.org/wp-content/uploads/2015/04/Estimating-Fact checkings-Effect.pdf

Ostermeier, E. (2011, February 10). Selection bias? PolitiFact rates republican statements as false at 3 times the rate of Democrats. Smart Politicshttps://editions.lib.umn.edu/smartpolitics/2011/02/10/selection-bias-politifact-rate/

Robertson, C. T., Mourão, R. R., & Thorson, E. (2020). Who uses fact checking sites? The impact of demographics, political antecedents, and media use on fact checking site awareness, attitudes, and behavior. The International Journal of Press/Politics25(2), 217-237. https://doi.org/10.1177/1940161219898055

Roth, Y., & Pickles, N. (May 11, 2020). Updating our approach to misleading information. Twitter.https://blog.twitter.com/en_us/topics/product/2020/updating-our-approach-to-misleading-information.html

Shin, J., & Thorson, K. (2017). Partisan selective sharing: The biased diffusion of fact checking messages on social media. Journal of Communication67(2), 233-255. https://doi.org/10.1111/jcom.12284

Stocking, G., Grieco, E., & Gottfried, J. (June 5, 2019). Partisans are divided on whether they associate the news media or Trump with ‘made-up’ news. Pew Research Center. https://www.pewresearch.org/fact-tank/2019/06/05/partisans-are-divided-on-whether-they-associate-the-news-media-or-trump-with-made-up-news/

Thorson, E. (2015). Belief echoes: The persistent effects of corrected misinformation. Political Communication33(3), 460-480. https://doi.org/10.1080/10584609.2015.1102187

Walker, M., & Gottfried, J. (2019, June 27). Republicans far more likely than Democrats to say fact checkers tend to favor one side. Pew Research Center. https://www.pewresearch.org/fact-tank/2019/06/27/republicans-far-more-likely-than-democrats-to-say-fact checkers-tend-to-favor-one-side/

Funding

The survey for this analysis was funded through internal funds from the Political Science Department at Western Kentucky University.

Competing Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics

The research protocol was approved by the institutional review board at Western Kentucky University. Human subjects gave informed consent at the beginning of the web survey before participating

Copyright

This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided that the original author and source are properly credited.

Data Availability

All materials needed to replicate this study are available at the International Public Opinion Lab (IPOL) website: http://www.timothysrich.com/ipol-research or via email at timothy.rich@wku.edu