Happiness and surprise are associated with worse truth discernment of COVID-19 headlines among social media users in Nigeria

---

Implications 

In the context of global uncertainty and anxiety about the COVID-19 pandemic, emotions may play a central role in responses to both true and false COVID-19-related headlines. This project asks whether emotions in general (and which distinct emotions) are associated with belief in, and intentions to read and share, COVID-19 misinformation. We test whether respondents in Nigeria have emotional reactions to false and true COVID-19 news headlines and examine whether an emotional response predicts greater belief in and intentions to read and share false, relative to true, headlines. 

Examining these questions in Nigeria, Africa’s most populous country and that with the largest number of Facebook users on the sub-continent,¹See https://www.internetworldstats.com/stats1.htm contributes to the understanding of global reactions to misinformation since existing research largely focuses on the U.S. and Europe. Critically, our data demonstrate that there is considerable need to improve social media users’ ability to discern true from false pandemic information. Even though our sample is highly educated—76% of our respondents had some university education—most respondents could not recognize false information. Specifically, 70% of respondents believed the majority of the false headlines they saw were true (see Figure A2 in Appendix A). This is a concerning statistic if the first step in curbing the spread of misinformation is correctly identifying false headlines from true headlines. However, several studies find little correlation between discernment—the ability to distinguish false from true information—and sharing of information (Guess et al., 2020; Pennycook et al., 2020; Pennycook et al., 2021). Among our respondents, even when they believed a story to be false, a minority were still interested in sharing it. Overall, 87% of respondents wanted to share at least one false news headline. Interventions designed to target both belief and sharing are necessary, especially during the pandemic as the spread of false COVID-19 “cures” can have devastating consequences.²See https://www.cnn.com/2020/03/23/africa/chloroquine-trump-nigeria-intl/index.html

The main goal of this study is to assess whether emotions are associated with belief in and intentions to read and share online (mis)information. We find that experiencing any emotional reaction is associated with worse truth discernment—i.e., greater belief in the accuracy of false, relative to true, headlines. We also find that experiencing emotions is associated with greater belief and interest in clicking to read and share headlines overall (equally for true and false headlines).

We also examine correlations between particular emotions and reactions to misinformation, which complements existing studies that examine the relationship between specific emotions and information consumption. Other studies find that distinct emotions are differentially associated with deliberation (Forgas, 2013; Holland et al., 2012; Lerner & Tiedens, 2006), which has been shown to affect discernment (Bago et al., 2020). Anxiety has been found to increase information seeking (Valentino et al., 2008). Anger has been shown to increase susceptibility to misinformation (Greenstein & Franklin, 2020; Han et al., 2020). Counter to these studies, we do not find robust results for anger and fear. Instead, we find that happiness and surprise are associated with worse discernment and greater sharing of false, relative to true, headlines. This departure from existing work could be due to differing samples, since most of the empirical literature comes from WEIRD populations,³WEIRD stands for Western, Educated, Industrialized, Rich, and Democratic (Henrich et al., 2010). and/or the fact that our study specifically focuses on COVID-19 information. 

Our findings have implications for policymakers, social media platforms, academics, and practitioners engaged in online digital literacy and campaigns to fight the spread of misinformation. We might expect emotions to predict belief and sharing behavior during public health crises, when emotions may be particularly intense. Yet there is a dearth of empirical evidence in this area, particularly from the Global South. Studies on the correspondence between emotions and discernment generally rely on U.S.-based samples (Martel et al., 2020). Given that fact-checking sites often advocate for regulating or suppressing emotions as a way to combat misinformation, the aim of our study is to understand the role emotions play in reactions to misinformation.

Our findings demonstrate that emotions are related to belief, clicking, and sharing of COVID-19-related information among social media users in Nigeria, and lend insights into containment and mitigation strategies in an important non-Western context. Future research should test interventions designed to regulate happiness and surprise, which may cloud one’s judgement and inhibit discernment. For instance, asking whether a headline evokes a particular emotion—such as happiness or surprise—may help individuals recognize these specific emotions. Light touch interventions that get users to identify their current emotional states might also help regulate particular emotions (Gross, 2002). Similarly, a prompt like “Just because a story makes you happy does not mean it’s true” could serve as a nudge to consider the accuracy of the headline (Pennycook et al., 2021). Furthermore, social media companies could use this information on the emotional reactions users have to posts (offered by their emoji reactions) to identify which posts might be more emotionally evocative, potentially more harmful, and then preferentially target these posts for fact-checking, flagging, and other mitigation strategies.

Given that respondent samples from Facebook in the Global South tend to be more urban and educated than average citizens (Rosenzweig et al., 2020), which may also reflect differential patterns of social media use in these contexts, it will be important to ensure that interventions are designed with these users in mind. In contexts where access to the internet and social media may be more recent, digital literacy trainings, which have been proven effective in other non-Western contexts may be important (Guess et al., 2020; Badrinathan, 2020). Such trainings could include techniques to spot emotionally evocative posts and could target social media users most susceptible to falling for misinformation.  

Finally, these findings have implications for tackling the ongoing infodemic and pandemic, by providing insights into which users are most vulnerable to misinformation. Our results demonstrate how social media users in Nigeria vary in terms of their susceptibility to false news, and where interventions should be focused—specifically on users who rely on intuition, younger users, and those who support the governing party. Similar to other studies with U.S. subjects, we find that cognitive reflection, measured by Cognitive Reflective Test (CRT), is positively associated with discernment (Pennycook & Rand, 2019). While identifying low-CRT individuals may be practically challenging, our results suggest that interventions could target younger social media users and ruling party supporters who are worse at discerning true from false COVID-19 information.⁴Age and political affiliation may be easier to identify since users have to report their age to Facebook, and many often follow political pages or join political groups. For instance, one “APC Nigeria” Facebook group has 40k members. Further research is required to understand whether the negative relationship we observe between governing party support and discernment persists even when another party controls the Presidency in Nigeria. Our findings suggest that partisanship is an important variable to pay attention to in the study of susceptibility to online misinformation, even in a context where party identification is relatively weak compared to the U.S. Further research is required to understand what kinds of interventions might help governing party supporters become better at truth discernment, and whether these would or would not have backlash effects among non-partisans and opposition supporters.

Given the existing cross-national evidence that susceptibility to COVID-19 misinformation is negatively associated with self-reported willingness to get vaccinated against the virus (Roozenbeek et al., 2020), these users might serve as the focal point for interventions targeting misinformation about the COVID-19 vaccine.

Findings

Finding 1: Emotionally evocative headlines are more likely to be believed, clicked, and shared.

When respondents have an emotional reaction to a headline, they are more likely to believe, want to read, and share that headline, regardless of whether it is true or false. Feeling any emotion is positively associated with our three outcomes of interest for all headlines (see Figure 1). Are emotions differentially associated with these outcomes based on the veracity of the headline? Meaning, are respondents more likely to believe, click, and share false relative to true headlines when they feel emotions overall and when they feel specific emotions?

In the study of misinformation, we not only care about the correlates of belief but also the correlates of discernment—i.e., the ability to distinguish true from false information. Here we find that emotional reactions are associated with worse truth discernment. Experiencing any emotion after reading a headline is associated with greater belief in false headlines relative to true headlines. Our respondents were better at discerning true from false news when they experienced no emotion after reading a headline.⁵This result is robust to the inclusion of control variables in our models, including age, gender, education, social media use, and support for the governing party (APC) (see Table C3 in Appendix C). We find no statistically significant differential relationship between emotion and clicking to read or sharing false headlines over true headlines. In other words, we cannot be sure that the differences we observe for reading and sharing outcomes are not simply due to chance.

Finding 2: Happiness and surprise are associated with worse discernment of headlines.

Examining each of the six distinct emotions, we find that half of these emotions are associated with increased belief, clicking to read, and sharing of any headline. Fear, happiness, and surprise are positively associated with our three outcomes overall, for both true and false headlines. Sadness is also positively correlated with overall outcomes, but this relationship is not statistically significant. Anger and disgust are negatively associated with overall belief in, clicking, and sharing of headlines. This last result may indicate that people use an “affect heuristic” when evaluating headlines and are less likely to believe a headline, regardless of its veracity, if their affective evaluation of the headline is negative—e.g., when they feel negative emotions, such as anger or disgust (Slovic et al., 2007). 

Analyzing each distinct emotion’s correspondence with discernment, we find that happiness and surprise are both negatively and significantly associated with truth discernment. While happiness is associated with increased belief in both true and false headlines, this relationship is stronger for false than true (see left panel in Figure 2). Surprise, on the other hand, is associated with both increased belief in false headlines and reduced belief in true headlines. Similarly, happiness and surprise are also associated with greater likelihood of wanting to share false headlines, compared to true headlines.⁶These results are consistent also when we include controls for age, gender, education, social media use, and APC support in our model (see Tables C4 and C5 in Appendix C). Section C.3 in Appendix C interacts APC support with our independent variables of interest (emotion and headline veracity) and finds consistent results. We do not find a robust relationship between these emotions, headline veracity, and clicking to read the story. The other four emotions we measured all have a positive correlation with discernment, as well as clicking and sharing more true, relative to false, headlines, but none of these relationships are statistically significant.⁷All regression tables are provided in Appendix B.

Finding 3: Older respondents, more reflective respondents, and respondents who do not support the governing party are better at discerning true from false COVID-19 information.

As displayed in Figure 3, we find that particular characteristics are significantly correlated with discernment of COVID-19 information among our sample of Nigerian Facebook users. Again, by discernment we mean the ability to distinguish true from false headlines, as evidenced by respondents saying they believe the true headlines are accurate and the false ones are not. We give each respondent a discernment score, which is the number of headlines they correctly identified as true or false (out of the 10 they saw) and use this as our outcome variable to see what respondent-level characteristics predict better discernment.

We find that reflecting or deliberating, rather than relying on intuition when making decisions, as measured by the Cognitive Reflection Test is significantly associated with truth discernment, as is being older and living in a mostly urban area (Frederick, 2005).⁸All of the demographic, social media use, cognitive reflection, and political variables we use to predict this discernment score were measured in our survey after the main task of presenting headlines, measuring emotional reactions, and our three outcome measures. The coefficient on urban has a p-value = .055. The positive correlation we observe between age and discernment is consistent with evidence from survey experiments with U.S. samples (Pennycook et al., 2021), but counter to the analysis of social media data among U.S. users (Grinberg et al., 2019). The positive association we observe between CRT scores and discernment is also consistent with existing research (Pennycook & Rand, 2019).

We also observe important negative correlations with discernment. First, we find that supporting the All Progressives Congress (APC), the current ruling party in Nigeria, is negatively correlated with discernment. We measured partisanship by first asking “do you feel close to any political party?” The 23% of respondents who answered “yes” were then asked, “Which party is that?” Options included All Progressive Congress (APC)-47%, Peoples’ Democratic Party (PDP)-48% and other-5%.⁹The percent of respondents who do not feel close to any party may seem low, but it is not uncommon in Nigeria. Here partisan affiliation tends to be weaker than say the U.S.—as is evident by the fact that about half (51%) of respondents in the most recent nationally representative Afrobarometer survey said they do not feel close to any political party (Afroba-rometer, 2018). Although partisanship is less entrenched in Nigeria, despite being a competitive democracy with two major parties like the U.S., it does not invalidate our finding that APC partisans are worse at discerning true from false headlines than other respondents. Second, religiosity is negatively correlated with discernment (p-value = .058). For partisanship, this correlation could be driven by greater trust in the media among ruling party supporters in Nigeria. More religious people, who might be more likely to hold views that God will protect believers from COVID-19, might exert less effort in truth discernment with respect to COVID-19 information because they feel protected.¹⁰Findings from a nationally representative phone survey released by NOI Polls in March 2020 revealed that 26% of Nigerians believed they are immune from COVID-19, and 40% of those people said it was because they are a “child of God.” (Source: https://noi-polls.com/covid-19-poll-result-release/) Other studies similarly observe a negative association between religiosity and discernment (Bronstein et al., 2019). Further research is required to understand the mechanism for these correlations.

Finally, we find that gender, education, social media use, and voting behavior are not statistically significant predictors of discernment. Recognizing that information about health crises may be particularly emotional, and quite distinct from general online misinformation, the results from this study should not be generalized to other types of misinformation.

Methods 

From April 1^_st to April 3^_rd, 2020, we conducted an online study of 1,341 Facebook users in Nigeria to understand the relationship between emotional reactions and belief in, reading intentions and sharing intentions of COVID-19-related misinformation. We had three research questions of interest. First, we investigated the relationship between emotions and our outcomes. Specifically, we wanted to know if emotional reactions to headlines were associated with greater belief in, reading, and sharing of false stories (compared to true stories). We selected these outcome measures following existing research on misinformation (Bode & Vraga, 2018; Guess et al., 2019; Pennycook et al., 2020; Tully et al.,2019), and also because they follow the logical process of reacting to a headline—first thinking about whether it is true, deciding whether to click to read it, and then sharing. Second, we examined these relationships for distinct emotions. Finally, we observed the individual characteristics that are associated with the ability to discern true from false headlines. 

This research was designed to test the generally assumed but seldom tested claim that emotionally evocative headlines are harder to decipher as true or false. Media outlets have suggested that people fall for spreading misinformation because it is emotionally evocative (Barr, 2019; Lunch, 2019). From psychology research, which suggests that deliberation improves discernment (Bago et al., 2020; Pennycook & Rand, 2019), we might also expect emotions to inhibit deliberation and therefore be associated with worse discernment. Drawing on these ideas, we prespecified two main hypotheses for this research.¹¹Our preanalysis plan is available on OSF (https://osf.io/vypfn/).

• H1: Respondents will be more likely to believe false headlines when they experience an emotional response. 
• H2: Respondents will be more likely to want to click to read and share false headlines when they have an emotional response.

Using Facebook advertisements, we recruited a sample of social media users in Nigeria aged 18 years old and older. Our sample is 68% male, 32% female, 76% have some university education, with the mean age of respondents being 28 years old. Though quite different from average citizens in Nigeria, this group of social media users—who clicked on an ad to take a survey—is a population of particular interest given the prevalence of misinformation on social media (Allcott et al., 2019; Wang et al., 2019). We focus on Nigeria because it is the most populous country in Africa, a media hub on the continent, and a prominent source of both fact-checking resources and an origin of COVID-19-related misinformation.¹²See https://www.poynter.org/coronavirusfactsalliance/

Facebook users who clicked on our ads were taken to a Qualtrics survey and shown media posts about COVID-19 that had appeared online. Respondents were asked a series of questions about these posts and then answered demographic questions, took the Cognitive Reflection Test (CRT),¹³This test is commonly used in psychology as a tool to measure an individual’s predisposition towards intuitive or “gut” reactions versus more reflective or deliberate decision making (Frederick, 2005). and reported their social media use. Respondents who completed the survey and provided their phone number were given ~1.50 USD in airtime sent to their mobile phone for their participation. 

We obtained the true media posts (“stimuli”) from major news outlets in Nigeria, including The Guardian, Daily Post, and Premium Times. The false stimuli came from COVID-19 stories that had appeared online but had subsequently been fact-checked (and verified as untrue) by AfricaCheck.org. All stimuli actually appeared online in Nigeria prior to the experiment. Each respondent saw 10 total COVID-19-related headlines, five true and five false, the order of which was random.¹⁴The 10 stimuli are presented in Appendix D. Headlines were a mix of stories about the state of the pandemic, travel, potential cures, and actions of and statements by political leaders.¹⁵We included a mix of political and non-political headlines to reflect the range of the types of headlines respondents would naturally be exposed to on social media concerning the pandemic, based on our own experience following media outlets in Nigeria on Twitter and Facebook at this time. At the end of the survey, respondents were informed which headlines were false and which were true.

After viewing each headline, respondents were asked to select any number of six emojis, each representing a distinct basic emotion (Ekman, 1992), to indicate how the headline made them feel (see Figure 4). Before the survey, we conducted a pretest with a separate sample of 76 Nigerians to confirm that the emojis were interpreted correctly. A majority of respondents were able to identify the emotion associated with each emoji.¹⁶Accuracy ranged from 55% for the “scared” emoji to 100% for “happy.” In the main survey, respondents indicated they felt surprise, fear, anger, disgust, happiness, sadness, or any combination of these emotions using these emojis (see Figure 4). The order of the emotions was randomized for each respondent. Respondents also had the option to report that they felt “neutral/no emotion.” If they felt any emotion after reading the headline, respondents were asked a follow-up question: “On a scale from 1 (not very strongly) to 7 (very strongly), how strongly did you feel [emotion]?”¹⁷The results from this measure of emotional strength are consistent with the main results presented here and are provided in Appendix C. 

Since relying on self-reported emotional states requires respondents to accurately understand and report their emotional reactions (Settle 2018), we would have ideally used an automatic or biological measure of emotions immediately after respondents read the headline. Because this was impossible in the context of an online survey, participants were randomly assigned to either give a response to the belief question or the emotion measure first after seeing the headline. We do this because studies show that labeling emotions can act as emotion regulation (Torre & Lieberman, 2018), reducing felt emotions. Asking about emotions after belief, however, is post-treatment with respect to this outcome. We observed no ordering effects in terms of the relationship between reported emotions and belief, and therefore pool data across these two conditions when analyzing the data (see Appendix C).

Belief was measured using the following question: “Do you think this headline accurately describes an event that actually happened?” We also asked whether respondents were (hypothetically) interested in clicking to read the headline and sharing the headline online. All responses were binary—respondents could either answer “yes” or “no.”

To analyze the correlation between emotional reactions and our outcomes of interest we use linear mixed-effects models.¹⁸Details of the model and tables presenting analyses for each emotion and all three outcomes are presented in Appendix B. These models account for the interdependence between observations because outcomes for the same respondent and for the same headline rated by each respondent are likely correlated. We include random effects for respondents and headlines because we are not interested in the specific influence of the particular individual or headline on the outcomes, rather we want to know whether emotions are correlated with the outcomes while controlling for the variation coming from specific headlines and respondents. We run a separate regression for each distinct emotion and interact the emotion dummy with the headline veracity indicator. 

Finally, we observe which types of respondents are more likely to believe false, rather than true, COVID-19 information, and assess the correlates of truth discernment among our sample. We use a linear regression to examine whether several demographic variables, as well as CRT scores, self-reported social media use, partisanship, religiosity, and past voting behavior predict discernment—the proportion of the 10 total headlines the respondent correctly identified as true or false (see Figure 3). In line with our pre-registered secondary hypothesis, we find that more reflective users, as measured by the Cognitive Reflection Test (CRT) are better at discerning true from false headlines.