Research note: Does the public support fact-checking social media? It depends who and how you ask

We analyze original survey data on support for social media companies’ fact-checking of politicians in general and President Trump in particular. We find overwhelming majorities of Democrats support factchecking in both instances, while a majority of Republicans support fact-checking of politicians in general but not of President Trump. Moreover, we find those concerned about contracting COVID-19 are more likely to support fact-checks. Rather than viewing fact-checks as a cure-all, we argue the importance of acknowledging their limitations and identifying when and on whom efforts may be effective.


Research note summary
• We conducted an experimental web survey via mTurk, an Amazon crowdsourcing platform, to identify support for fact-checking politicians and President Trump separately. We then employed statistical analysis to identify differences by party identification and other demographic factors. • We find that majorities supported fact-checking with clear differences among Democrats and Republicans. Democratic support shows marginal differences between fact-checks of politicians versus President Trump. A majority of Republicans support fact-checks of politicians but not of Trump. • Concern about contracting COVID-19 corresponds with support for fact-checks with a larger effect among Republicans. • Fact-checking interventions work only for those people who see value in fact-checking.

Implications
Donald Trump has notably made thousands of factually inaccurate statements on Twitter. In May of 2020, for the first time, Twitter fact-checked a Trump tweet on mail-in voting (Mangan & Brueninger 2020). Twitter also recently put labels on a selection of Trump's tweets that violated company policies (Feiner 2020). Facebook, originally reluctant to act, announced in June that it would add labels to posts on voting directing users to credible election information (Mihalcik 2020). These actions sparked new questions about the role of social media companies in fact-checking statements by politicians. Such debates have increased importance due to the COVID-19 pandemic as medical misinformation can result in prolonging the pandemic and increasing the number of deaths. Social media fact-checks can take multiple forms, from Facebook collaborating with a third-party organization to label misinformation and post fact-checks under such posts (Bode & Vraga, 2015) to Twitter's internal methods to categorize posts as misleading, disputed, or unverified (Roth & Pickles, 2020). Who should perform fact-checks and in what capacity further shapes perceptions, and perhaps perceived biases, of the process? Table 2 provides a snapshot of current fact-checking policies and procedures among four major social media platforms. Facebook Facebook identifies misinformation after it goes viral and refers it to third party fact-checkers before applying a label. Punishments are threatened for repeat offenders. (Facebook)

Twitter
No comprehensive misinformation policies exist. Twitter will label or remove manipulated media, or anything intended to misinform or interfere with elections or other civic processes. Exceptions are made for key elected officials.
YouTube YouTube provides information from third-party fact-checkers on specific searches. Specific videos are not fact-checked. (Google)

Pinterest
Pinterest says they will remove content from conspiracy theories, medical misinformation, election misinformation, and disinformation campaigns. Manipulated media will also be removed. The fact-checking process is not outlined. (Pinterest) Experimental research commonly finds a positive relationship between support for fact-checking and political knowledge (e.g. Thorson 2016;Fridkin et al. 2015 Robertson et al. 2020;Lyons et al. 2020). Shin and Thorson (2017) find not only selective sharing of fact-checks based on partisan preferences, but hostility towards outgroup fact-checks, especially among Republicans. This may be due to analyses of fact-checks labeling Republican politicians' claims false at a higher rate than Democratic counterparts (e.g. Center for Media and Public Affairs at George Mason University 2013; Ostermeier 2011). Thus, fact-checks may only influence those that already see value and credibility in the factchecking process.
Despite the positive normative implications of fact-checking, we expect differences of opinion on factchecking based on partisanship and whether politicians or Trump specifically were targets of fact-checks. In addition, the role of misinformation regarding COVID-19 (e.g. Krause et al. 2020) may increase support for fact-checks. For example, President Trump claimed that 99% of COVID-19 cases were harmless (Dugyala & Forgey 2020) and made 654 false statements (Dale & Subramanian 2020) about the pandemic during the first 14 weeks of the pandemic by CNN's count.
Despite noble intentions, there is little evidence that fact-checks change support for candidates (e.g. Nyhan et al. 2019;Barrera et al. 2020), and instead could lead Republicans to disregard or dismiss efforts as partisan attacks. This could produce more partisan rancor and cause fact-checking to be counterproductive in attempting to improve general political discourse. Our evidence suggests that for some people, fact-check efforts of politicians may never work. However, particular concerns about health and safety, such as contracting COVID-19, may increase interest in social media fact-checks. However, fact-checks alone, even if limited to public health issues, likely are not enough to correct misinformation if the sources and impartiality of the fact-checks themselves are in dispute.

Findings
This study investigates fact-checks conducted internally by social media platforms and support for these efforts based on partisanship, the target of fact-check efforts, and concern for contracting COVID-19. This includes three important findings on the extent of support for fact-check efforts.
Finding 1: Democrats are more supportive of fact-checks overall, while support among Republicans drops when the focus is on Donald Trump. Figure 1 shows clear differences between how Democratic and Republican respondents evaluate the two versions. For visual clarity, we recoded the five-point scale into three categories by combining "strongly disagree" with "disagree" and "strongly agree" with "agree".
The results show Democrats as more supportive of fact-checks overall than Republicans. More interesting is that Republican support for fact-checking Trump dropped by 15 percentage points in comparison to fact-checking politicians in general, with a difference in means test statistically significant at the .01 level. One potential reason for the decline in support is that Republicans may interpret the prompt as unfairly singling out Trump, while supporting fact-checks of politicians of politicians that they do not like.
Similarly, because Republicans tend to exhibit greater positive affect for their party than Democrats, the perception of Republicans being fact-checked may produce defensive reactions. Trump's status as an unconventional politician may also be motivating such variation among Republicans. Surprisingly, Democrats' support for fact-checking did not increase when specifying President Trump in the statement. This may be because Trump was already on the minds of those presented with the "politicians" version of the statement since he tends to be fact-checked by the media daily (Kessler et al. 2020).

Figure 1. Support for fact-checks of politicians and President Trump.
Finding 2: Those concerned about contracting COVID-19 statistically were more likely to support factchecks.
We found that people who were concerned about contracting COVID-19 statistically were more likely to support fact-checks, regardless of which fact-check statement they received. Overall, 65.04% of Democrats were worried about contraction (a four or five on a five-point scale), compared to only 42.7% of Republicans. Figure 2 below separates respondents by their concern for contracting COVID-19 and support for fact-checks under each version. Overall, we see support for fact-checks of politicians and Trump in particular are roughly 36% higher among those concerned about COVID-19. Among Democrats concerned about COVID-19, we see a larger increase for fact-checking politicians (24.66%) compared to Trump (15.17%), while among Republicans we see the opposite, with COVID-19 concerns increasing support for fact-checks of Trump (42.84%) compared to politicians more broadly (31.11%).

Finding 3: The effects of the framing of fact-checks and concern about contracting COVID-19 remain after considering demographic factors and ideological extremity.
We ran separate ordered logistic regression models on support for social media fact-checks for Democrats and Republicans. The models included whether respondents received the Trump version of the fact-check statement, along with basic demographic factors (gender, age, education, and household income), ideological extremity, and concern about contracting COVID-19. For Democrats, those receiving the Trump version corresponded with greater support for fact-checks (.35 on the five-point scale), significant at p = .05, although with wide confidence intervals. Age also showed a small positive influence (.03) on fact-check support (p < .001), with ideological extremity having the largest positive influence (.58, p <.001). Lastly, concern over COVID-19 corresponds with .32 point increase in support for fact-checks, significant at p = .001.
Predicted probabilities find that Democrats that received the Trump version were more likely to support fact-checks (90% vs. 86%). In addition, those most worried about contracting COVID-19 (a five on the five-point scale) were 14-18% more supportive of fact-checks than the least worried (politicians version: 90% vs. 72%; Trump version 93% vs. 79%).
For Republicans, several factors were statistically significant, most notably that receiving the Trump version corresponded with the largest decline in support (-.75 on a five-point scale, p =.001), again with wide confidence intervals. Women showed the largest positive correlation with fact-checking (.58, p = .008), while age had a minor substantive impact (-.16, p =.009). Unlike Democrats, ideological extremity was not statistically significant among Republicans. Finally, and similar to Democrats, concerns about contracting COVID-19 corresponded with a support for fact-checks (.42, p <.0001).
Predicted probabilities for fact-checks show that Republicans receiving the Trump version were less supportive of fact-checks by 18% (politicians version: 58%, Trump version: 39%). Like Democrats, those most worried about contracting COVID-19 were more supportive of fact-checks than the least worried, however here the variation is 38-39% (politicians version: 75% vs. 36%; Trump version 59% vs. 21%). Put another way, while Republican support for fact-checks lagged behind that of Democrats, we find that concern for contracting COVID-19 actually exhibits a larger substantive effect on support for fact-checks among Republicans.

Methods
We asked our study participants three questions: does the public support social media fact-checking, does public support differ when focused on politicians in general versus President Trump specifically, and does concern about contracting COVID-19 influence support for fact-checking? To test our questions and isolate the views of fact-checks on politicians versus President Trump, we used an original web survey as we could randomly assign respondents differing versions of our fact-check question. We relied on an original web survey, recruiting 1,027 American respondents 18-79 years of age via mTurk on July 7, 2020. Participants were on average 40 years old, 55% of which were male, and largely educated, with 65% having at least a Bachelor's degree. This method of recruitment allowed for a regionally, ideologically, and ethnically diverse sample. mTurk recruitment is also commonplace in experimental research in social science.
We asked respondents first a series of demographic questions, across several pages, and respondents could not change their answers after leaving a page. We then randomly assigned respondents to one of two statements and asked them to rank their support for fact-checking either Trump or politicians on a five-point Likert scale ("strongly disagree" to "strongly agree").
• Version 1: Social media companies should provide fact checks of statements from politicians.
• Version 2: Social media companies should provide fact checks of statements from President Trump. Besides our experimental question addressed earlier, we ask respondents to evaluate the following statement on a 1 to 5 scale: "I am concerned about contracting ." This question allows us to look for potential relationships between fear of COVID-19 and support for fact-checks on social media.
We followed descriptive data with ordered logit regression analysis in order to identify the size of the effect of framing fact-checks in a generic fashion versus on a particular politician, controlling for potentially important demographic factors. Ordered logit models, rather than ordinary least squares (OLS) regressions are justified when the dependent variable is a Likert scale. Running separate models by partisan identification, we included controls for gender, age, household income, education, ideological extremity (how far away from a moderate position of a four on a seven-point ideology scale), and concern about contracting COVID-19. We then generated predicted probabilities for support for fact-checking (a four or five on our five-point scale) at different levels of concern for COVID-19 and for the fact-check prompt received, holding all other independent variables at their mean. This allows us to identify the substantive effect of COVID-19 concerns on support for fact-checking, ultimately showing a twenty percent or greater increase in the likelihood of support for fact-checks as one moves from not concerned to concerned about contraction.

Limitations and robustness
We use an experimental design embedded in a web survey to identify how partisanship and concern about COVID-19 influence views of fact-checking; however, several caveats are necessary. First, our recruitment via mTurk is a convenience sample, and while appropriate for experimental randomizations, does not perfectly mirror the national population. This may affect generalizability. Secondly, our research design only tests one specific politician, Donald Trump, and not others. For example, we cannot determine whether Democrats would be less supportive of fact-checks or Republicans more supportive if the politician mentioned was a Democrat (e.g. Joe Biden). Future work should include such tests, as well as testing whether the public elicits greater support of fact-checks based on the gender, experience, or level of office of the politician.
We did not provide additional information about social media fact-checks, so it is possible that partisans interpreted the basic question differently and counter to the intention of the experiment. Nor can we fully parse out pre-existing and often partisan beliefs on the source of misinformation (e.g. Stocking, Grieco, & Gottfried, 2019) or reliable media outlets (e.g. Nielsen & Graves, 2017). Partisans may also have divergent baseline views of both social media and fact-checks in general, which admittedly cannot be disaggregated in this research design. Lastly, we cannot parse out support for different forms of fact-checks (e.g. internal or third-party efforts). Barrera, O., Guriev, S., Henry, E., & Zhuravskaya, E. (2020). Facts, alternative facts, and fact checking in times of post-truth politics.