Do the right thing: Tone may not affect correction of misinformation on social media

Implications 

Research increasingly shows that correcting misinformation is effective at getting people to update their beliefs—when you give them new facts, they tend to reduce their beliefs in misinformation (Porter & Wood, 2019; Walter & Murphy, 2018). This works in a variety of contexts, including on social media, where misinformation often spreads quickly but can also effectively be corrected, including by other social media users (Bode & Vraga, 2018; Vraga & Bode, 2017; 2018). This is particularly promising, given that the visible and networked nature of social media allows for observational correction—users on social media can watch others be corrected (Vraga & Bode, 2017), increasing the impact of any given correction as it is seen by an entire network of users. 

In this study, we test two open questions from this line of research. First, does the tone of a correction on social media—specifically, whether users adopt a neutral tone, in which the correction posts simply convey a fact, an uncivil tone, in which the correction posts insult the original poster, or an affirmative tone, in which correction posts show empathy and affirm the original poster, when responding to misinformation—increase or decrease its ability to reduce misperceptions (Chen, 2017; Ecker, Swire, & Lewandowsky, 2014)? And second, can users correct related topics of misinformation, not mentioned in the original post, while correcting a direct claim made by a social media user?  

Perhaps our clearest takeaway is that tone does not influence how effective corrections are. In other words, a correction that focuses purely on offering a factual response to misinformation is equally successful in reducing misperceptions as one that is uncivilly attacking the intelligence of the person sharing the misinformation. For this reason, ‘fixing’ the problem of incivility on social media, while it may be admirable on its own, should not be expected to impact the problem of misinformation and misperceptions resulting from social media. Although uncivil replies do not make corrections less effective, the tone changes perceptions of the original tweet making it appear more uncivil. In other words, when people see an uncivil reply, they perceive it as such and also think the original post was more uncivil, suggesting a possible spillover effect (Anderson, Brossard, Scheufele, Xenos, & Ladwig, 2014). Although uncivil corrections are equally effective in reducing misperceptions, they may negatively influence people’s perceptions of the broader dialogue, which may prove consequential for other outcomes beyond correction.

Likewise, an affirmative tone—which acknowledges and affirms that confusion or uncertainty on the issue is understandable—is neither more nor less effective in reducing misperceptions than a neutral or uncivil tone. While an affirmative tone may make the correction more palatable for the person getting corrected or for the individual engaging in the correction, it may not affect how the broader social media community witnessing the correction interprets or accepts the corrective information. It is worth noting that although affirmation is not simply an abundance of civility, our participants did not see the affirmative correction as more civil than a neutral correction—both were perceived as civil. Future research should examine whether stronger affirmations are perceived as more empathetic and less threatening than other types of corrections, which we did not test directly, and if this leads to more success in reducing misperceptions.

These findings have several implications. First, while we do not encourage social media users to be rude, uncivil corrections that include the same factual information are not less effective at reducing misperceptions. Thus, if a social media user feels more comfortable correcting with a somewhat rude reply, they should not worry that doing so will make their reply less effective in reducing misperceptions among bystanders seeing the interaction. Likewise, affirmative corrections do not appear to make audiences seeing the interaction—who may hold misperceptions themselves—more receptive to the correction. If being empathetic and affirming makes offering a correction easier, users should feel empowered to do so. Altogether, this suggests that those engaging in correction on social media have the flexibility to adopt the tone they think most appropriate. Users might consider the target of the misinformation, community norms, or their own comfort when selecting what approach to take in responding to misinformation (Tandoc, Lim & Ling, 2020). For those creating interventions to increase corrective efforts on social media, like media literacy groups and professional fact checkers, greater attention should be given to the content of the message—including links and supportive information (Vraga & Bode, 2018)—as compared to the tone of that content, to maximize the volume and effectiveness of corrections. Journalists, public health authorities, or other experts can also employ these strategies—sharing credible and relevant information in direct response to user misinformation on social media. 

Of course, there is potential for the tone of a correction to affect other outcomes beyond misperceptions, which we do not explicitly test. If an uncivil tone leads to disengagement with the issue or an affirmative tone makes it more likely users are willing to engage in correction of those spreading misinformation change their mind on the issue, this could change recommendations on appropriate tone to be used.  Likewise, an uncivil tone may be more problematic (or an affirmative tone more successful) for more emotional, salient, politicized, or partisan issues (Bolsen & Druckman, 2018), which future research should test. 

Opportunities should also be taken to preemptively target related misperceptions when correcting on a particular topic. Essentially this allows correction to be doubly effective—correcting the specific piece of misinformation espoused on social media, but also a piece of related misinformation at the same time. In this study, the misinformation post only discussed the nutritional value of unpasteurized milk. However, public health officials tend to be more concerned with the health risks of unpasteurized milk (CDC, n.d.). Correcting related misperceptions – in our case, about the health risks—can therefore encourage appropriate behaviors based on more complete information. This preemptive correction may be particularly effective when multiple topics of misinformation are prominent for a given topic, as is often the case. More specifically, this might mean that social media platforms pair multiple corrections, rather than just surfacing one most directly related to misinformation a user has posted. For example, Facebook currently shows users related articles with fact checks when they see a misinformation post that has been identified as such by third-party fact checkers (Lyons, 2017). They might consider adding a second related article, debunking a related myth, in order to maximize the corrective effects on users. 

Findings

Finding 1: User corrections decrease people’s belief in misinformation regarding the nutritional value of raw milk, and this is true for all corrections regardless of tone

Participants who saw any of the corrections expressed lower levels of misperceptions that raw milk is more nutritious than pasteurized milk as compared to people who saw the misinformation without any correction. Moreover, there are no differences in misperceptions about raw milk’s nutritional value among people who saw a neutral (factual-only) correction, an uncivil correction, or an affirmative correction.

Finding 2: Corrections also reduce people’s beliefs in a related piece of misinformation (that raw milk is safe to drink), and this is true for all corrections

The original misinformation post did not mention the safety of raw milk, nor did the first correction. Instead, this idea only emerged in the second correction, which not only debunked the original misinformation but also added that pasteurized milk is safer to drink than raw milk. Our analyses confirm that exposure to any of the corrections (regardless of tone) reduced misperceptions regarding raw milk’s safety as compared to people who only viewed misinformation. Moreover, there are no differences in terms of raw milk misperceptions among the three corrections themselves; misperceptions in terms of raw milk’s safety are equally low regardless of the tone of the correction.

Finding 3. Participants thought the uncivil corrections were uncivil

When rating the civility of the corrections, the uncivil corrections were seen as more uncivil than either the affirmative or neutral corrections, which were seen as equally civil. However, even the uncivil corrections were only seen as moderately uncivil, receiving a score of 4.08 out of a possible 7 in terms of the civility of the corrections. Notably, the affirmative corrections were not seen as more civil than the neutral corrections: both were seen as more civil than not and as more civil than the uncivil correction. However, this measure (perceived civility) is not designed to capture affirmation which is about empathy and threat reduction, so we are cautious in our interpretation.  

Finding 4. Uncivil corrections led people to believe the original tweet was also more uncivil, even though it was not

The original misinformation tweet, which was consistent across all the content seen by participants, was seen as more uncivil when the corrections were uncivil than when that misinformation tweet appeared absent any corrections. Meanwhile, the neutral and affirmative corrections fell between these extremes in terms of perceptions of their civility. In general, however, people did not perceive any of the tweets or replies to be particularly uncivil.

Methods

An experiment is the best way to test our hypotheses and research questions because it lets us isolate the specific effects of the tone of correction on misperceptions of those seeing the interaction. Participants were recruited from Amazon’s Mechanical Turk in September of 2018. Participants were an average of 36 years old, 55% male, and largely educated (52% had at least a Bachelor’s degree).

After completing a short pre-test questionnaire, participants were randomly assigned to view one of four simulated Twitter feeds (N=610).¹We examine four of the 10 total conditions we fielded in this article, excluding a pure control condition, in which participants were not exposed to any misinformation or correction, as well as 5 conditions that included a second manipulation focused on news literacy. None of the conditions tested in this article included news literacy interventions.  In each condition (that is, each group into which participants were randomly assigned), participants were instructed to read a page of Twitter posts as if they were viewing their own feed, which we said were taken from someone’s feed. The simulated feed contained six Twitter posts, including one manipulated post and five posts validated as politically neutral and plausible social media posts (Authors, 2016). Participants were required to spend 15 seconds on the page of posts before they could continue with the survey. After answering a series of questions regarding their experience with the feed, their evaluation of the posts, and their attitudes towards the target issue (raw milk), participants were thanked for their participation and debriefed. Debriefing included a statement that pasteurized milk is equally nutritious and safer to consume than raw (unpasteurized) milk and provided a link to more information from the CDC. Participants were paid $1.10 for participating in the survey. 

The third post on each feed contained the experimental manipulation. In all conditions, the same user posted a meme (an image macro featuring Morpheus from the Matrix) claiming that pasteurizing milk “kills the nutrients in raw milk” (see Figure 5). We selected this myth for several reasons. First, it is increasingly relevant, as raw milk is gaining attention and policy protections around the U.S. in recent years (Rahn, Gollust, & Tang, 2017). Second, the specific myth related to relative nutrition of raw and pasteurized milk is prominently debunked—the CDC and the FDA both have corrected this myth in their online materials (e.g., CDC, n.d.; FDA, n.d.). While overall consumption levels of raw milk (1.8% in the last seven days among respondents in the Midwest, Rahn, et al., 2017) and raw cheese (6.9% in the last seven days among respondents in the Midwest, Rahn, et al., 2017) are quite low, such consumption contributes to an outsized number of food-borne illness in the U.S. (60% of those reported in relation to dairy products were connected to unpasteurized milk according to Langer, et al., 2012). There is also at least moderate support for loosening restrictions on selling unpasteurized milk (52% supported loosening restrictions in a survey of Midwesterners, Rahn, et al., 2017). 

In the misinformation-only condition, there were no responses to this post. In all correction conditions, two users responded to the post to debunk the misinformation, while providing links²Note that because they are simulated feeds, participants did not have the opportunity to click the links. No additional factual content is therefore associated with the presence of the links. to expert sources (e.g. the CDC and FDA), in keeping with best practices (e.g., Vraga & Bode, 2018). The first response included a link to the CDC and directly responded to the misinformation that pasteurization kills nutrients. The second response reinforced the equal nutritional value of raw and pasteurized milk and added that pasteurization keeps people from getting sick from bacteria in milk, providing a link to the FDA. This second claim is what we use to test whether ‘related misperceptions’ of raw milk safety are updated.

In the “neutral correction” condition, the responses provide a factual response and mimic the tone of earlier designs on observational correction (e.g., Bode & Vraga, 2018; Vraga & Bode, 2017; 2018). The specific text of the first reply was “This isn’t true. Pasteurizing milk doesn’t affect its nutrients. cdc.gov/foodsafety/raw…,” and the second reply said: “Pasteurization does not affect milk’s nutrients, and it keeps people from getting sick from bacteria in raw milk. fda.gov/ForConsumers/C.” (Figure 6). In the “affirmative correction,” the responses attempt to validate the participants’ worldview and concerns, expressing understanding of possible “confusion” on the issue, as suggested by Lewandowsky et al. (2012). The text of the first affirmative reply said “I know it can be super confusing, but this isn’t true. Pasteurization doesn’t affect the nutrients in milk at all. cdc.gov/foodsafety/raw . . .” and the second reply said “This is such a scary thought, but I just learned that pasteurization does not affect milk’s nutrients, and it keeps people from getting sick from bacteria in raw milk! fda.gov/ForConsumers/C.” Finally, the “uncivil correction,” includes insults and name-calling (Chen, 2017) towards the original poster in the correction, telling the poster not to be “stupid” and calling them an “idiot”. The first uncivil reply said “Oh come on, don’t be stupid. Everyone knows that pasteurizing milk doesn’t affect its nutrients. cdc.gov/foodsafety/raw . . .” and the second reply said: “I can’t believe how dumb this is. Pasteurization does not affect milk’s nutrients, and it keeps people—even idiots like you—from getting sick from bacteria in raw milk. fda.gov/ForConsumers/C.”Although this language could be considered a mild form of incivility (Chen, 2017), especially given the kind of language that circulates on Twitter (Oz et al., 2018; Phillips, 2015), it clearly attacks the Twitter user directly and is used as a means of undercutting their claim about raw milk (which in fact is incorrect).

To test the effects on misperceptions, we perform a series one-way ANOVAs, first comparing exposure to any correction (the three corrections combined) versus misinformation-only; then comparing among the three correction conditions. For the effects on civility perceptions, we compare between all four (original tweet civility) or three (correction civility) conditions using an omnibus test. For significant results, we use pairwise comparisons with a Bonferroni correction to adjust for additive error to examine differences between the conditions. Figures include the estimated marginal means for all relevant conditions to facilitate comparison. See the supplemental methods addendum for more information.