Happiness and surprise are associated with worse truth discernment of COVID-19 headlines among social media users in Nigeria

Do emotions we experience after reading headlines help us discern true from false information or cloud our judgement? Understanding whether emotions are associated with distinguishing truth from fiction and sharing information has implications for interventions designed to curb the spread of misinformation. Among 1,341 Facebook users in Nigeria, we find that emotions—specifically happiness and surprise—are associated with greater belief in and sharing of false, relative to true, COVID-19 headlines. Respondents who are older are more reflective, and do not support the ruling party are better at discerning true from false COVID-19 information.

• Respondents are more likely to believe, want to read and share headlines (regardless of veracity) when they feel any emotion. Emotional responses are associated with worse truth discernment and the ability to distinguish true from false headlines when assessing belief (but not reading or sharing). We find that happiness and surprise, in particular, are associated with believing and sharing false, relative to true, headlines. • Interventions to improve discernment of COVID-19 information should target youth, those who rely on intuition, and ruling party supporters in Nigeria. • Understanding the role emotions play in reactions to misinformation has implications for technology platforms, governments, and citizens interested in combating the COVID-19 "infodemic." Future research should test the causal relationship between emotions and belief in COVID-19 misinformation and interventions designed to regulate specific emotions in diverse settings.

Implications
In the context of global uncertainty and anxiety about the COVID-19 pandemic, emotions may play a central role in responses to both true and false COVID-19-related headlines. This project asks whether emotions in general (and which distinct emotions) are associated with belief in, and intentions to read and share, COVID-19 misinformation. We test whether respondents in Nigeria have emotional reactions to false and true COVID-19 news headlines and examine whether an emotional response predicts greater belief in and intentions to read and share false, relative to true, headlines. Examining these questions in Nigeria, Africa's most populous country and that with the largest number of Facebook users on the sub-continent, 2 contributes to the understanding of global reactions to misinformation since existing research largely focuses on the U.S. and Europe. Critically, our data demonstrate that there is considerable need to improve social media users' ability to discern true from false pandemic information. Even though our sample is highly educated-76% of our respondents had some university education-most respondents could not recognize false information. Specifically, 70% of respondents believed the majority of the false headlines they saw were true (see Figure A2 in Appendix A). This is a concerning statistic if the first step in curbing the spread of misinformation is correctly identifying false headlines from true headlines. However, several studies find little correlation between discernment-the ability to distinguish false from true information-and sharing of information (Guess et al., 2020;Pennycook et al., 2021). Among our respondents, even when they believed a story to be false, a minority were still interested in sharing it. Overall, 87% of respondents wanted to share at least one false news headline. Interventions designed to target both belief and sharing are necessary, especially during the pandemic as the spread of false COVID-19 "cures" can have devastating consequences. 3 The main goal of this study is to assess whether emotions are associated with belief in and intentions to read and share online (mis)information. We find that experiencing any emotional reaction is associated with worse truth discernment-i.e., greater belief in the accuracy of false, relative to true, headlines. We also find that experiencing emotions is associated with greater belief and interest in clicking to read and share headlines overall (equally for true and false headlines).
We also examine correlations between particular emotions and reactions to misinformation, which complements existing studies that examine the relationship between specific emotions and information consumption. Other studies find that distinct emotions are differentially associated with deliberation (Forgas, 2013;Holland et al., 2012;Lerner & Tiedens, 2006), which has been shown to affect discernment (Bago et al., 2020). Anxiety has been found to increase information seeking (Valentino et al., 2008). Anger has been shown to increase susceptibility to misinformation (Greenstein & Franklin, 2020;Han et al., 2020). Counter to these studies, we do not find robust results for anger and fear. Instead, we find that happiness and surprise are associated with worse discernment and greater sharing of false, relative to true, headlines. This departure from existing work could be due to differing samples, since most of the empirical literature comes from WEIRD populations, 4 and/or the fact that our study specifically focuses on COVID-19 information.
Our findings have implications for policymakers, social media platforms, academics, and practitioners engaged in online digital literacy and campaigns to fight the spread of misinformation. We might expect emotions to predict belief and sharing behavior during public health crises, when emotions may be particularly intense. Yet there is a dearth of empirical evidence in this area, particularly from the Global South. Studies on the correspondence between emotions and discernment generally rely on U.S.-based samples (Martel et al., 2020). Given that fact-checking sites often advocate for regulating or suppressing emotions as a way to combat misinformation, the aim of our study is to understand the role emotions play in reactions to misinformation.
Our findings demonstrate that emotions are related to belief, clicking, and sharing of COVID-19related information among social media users in Nigeria, and lend insights into containment and mitigation strategies in an important non-Western context. Future research should test interventions designed to regulate happiness and surprise, which may cloud one's judgement and inhibit discernment. For instance, asking whether a headline evokes a particular emotion-such as happiness or surprise-may help individuals recognize these specific emotions. Light touch interventions that get users to identify their current emotional states might also help regulate particular emotions (Gross, 2002). Similarly, a prompt like "Just because a story makes you happy does not mean it's true" could serve as a nudge to consider the accuracy of the headline (Pennycook et al., 2021). Furthermore, social media companies could use this information on the emotional reactions users have to posts (offered by their emoji reactions) to identify which posts might be more emotionally evocative, potentially more harmful, and then preferentially target these posts for fact-checking, flagging, and other mitigation strategies.
Given that respondent samples from Facebook in the Global South tend to be more urban and educated than average citizens (Rosenzweig et al., 2020), which may also reflect differential patterns of social media use in these contexts, it will be important to ensure that interventions are designed with these users in mind. In contexts where access to the internet and social media may be more recent, digital literacy trainings, which have been proven effective in other non-Western contexts may be important (Guess et al., 2020;Badrinathan, 2020). Such trainings could include techniques to spot emotionally evocative posts and could target social media users most susceptible to falling for misinformation.
Finally, these findings have implications for tackling the ongoing infodemic and pandemic, by providing insights into which users are most vulnerable to misinformation. Our results demonstrate how social media users in Nigeria vary in terms of their susceptibility to false news, and where interventions should be focused-specifically on users who rely on intuition, younger users, and those who support the governing party. Similar to other studies with U.S. subjects, we find that cognitive reflection, measured by Cognitive Reflective Test (CRT), is positively associated with discernment . While identifying low-CRT individuals may be practically challenging, our results suggest that interventions could target younger social media users and ruling party supporters who are worse at discerning true from false COVID-19 information. 5 Further research is required to understand whether the negative relationship we observe between governing party support and discernment persists even when another party controls the Presidency in Nigeria. Our findings suggest that partisanship is an important variable to pay attention to in the study of susceptibility to online misinformation, even in a context where party identification is relatively weak compared to the U.S. Further research is required to understand what kinds of interventions might help governing party supporters become better at truth discernment, and whether these would or would not have backlash effects among non-partisans and opposition supporters.
Given the existing cross-national evidence that susceptibility to COVID-19 misinformation is negatively associated with self-reported willingness to get vaccinated against the virus (Roozenbeek et al., 2020), these users might serve as the focal point for interventions targeting misinformation about the COVID-19 vaccine.

Findings
Finding 1: Emotionally evocative headlines are more likely to be believed, clicked, and shared.
When respondents have an emotional reaction to a headline, they are more likely to believe, want to read, and share that headline, regardless of whether it is true or false. Feeling any emotion is positively associated with our three outcomes of interest for all headlines (see Figure 1). Are emotions differentially associated with these outcomes based on the veracity of the headline? Meaning, are respondents more likely to believe, click, and share false relative to true headlines when they feel emotions overall and when they feel specific emotions?
In the study of misinformation, we not only care about the correlates of belief but also the correlates of discernment-i.e., the ability to distinguish true from false information. Here we find that emotional reactions are associated with worse truth discernment. Experiencing any emotion after reading a headline is associated with greater belief in false headlines relative to true headlines. Our respondents were better at discerning true from false news when they experienced no emotion after reading a headline. 6 We find no statistically significant differential relationship between emotion and clicking to read or sharing false headlines over true headlines. In other words, we cannot be sure that the differences we observe for reading and sharing outcomes are not simply due to chance.

Figure 1. Rate of belief in, clicking, and sharing of headlines by true/false headline veracity and respondents' emotional reaction. Blue bars indicate headlines where respondents reported feeling no emotion at all (neutral). Orange bars indicate headlines where respondents reported feeling any of the six distinct emotions presented (anger, fear, sadness, happiness, surprise, and disgust). Error bars represent 95% confidence intervals.
Finding 2: Happiness and surprise are associated with worse discernment of headlines.
Examining each of the six distinct emotions, we find that half of these emotions are associated with increased belief, clicking to read, and sharing of any headline. Fear, happiness, and surprise are positively associated with our three outcomes overall, for both true and false headlines. Sadness is also positively correlated with overall outcomes, but this relationship is not statistically significant. Anger and disgust are negatively associated with overall belief in, clicking, and sharing of headlines. This last result may indicate that people use an "affect heuristic" when evaluating headlines and are less likely to believe a headline, regardless of its veracity, if their affective evaluation of the headline is negative-e.g., when they feel negative emotions, such as anger or disgust (Slovic et al., 2007).
Analyzing each distinct emotion's correspondence with discernment, we find that happiness and surprise are both negatively and significantly associated with truth discernment. While happiness is associated with increased belief in both true and false headlines, this relationship is stronger for false than true (see left panel in Figure 2). Surprise, on the other hand, is associated with both increased belief in false headlines and reduced belief in true headlines. Similarly, happiness and surprise are also associated with greater likelihood of wanting to share false headlines, compared to true headlines. 7 We do not find a robust relationship between these emotions, headline veracity, and clicking to read the story. The other four emotions we measured all have a positive correlation with discernment, as well as clicking and sharing more true, relative to false, headlines, but none of these relationships are statistically significant. 8 7 These results are consistent also when we include controls for age, gender, education, social media use, and APC support in our model (see Tables C4 and C5 in Appendix C). Section C.3 in Appendix C interacts APC support with our independent variables of interest (emotion and headline veracity) and finds consistent results. 8 All regression tables are provided in Appendix B.

Figure 2. Rate of belief in true/false headlines for happiness and surprise. Blue bars indicate headlines where respondents reported feeling that particular emotion. Orange bars indicate headlines where respondents reported not feeling that particular emotion. Error bars represent 95% confidence intervals.
Finding 3: Older respondents, more reflective respondents, and respondents who do not support the governing party are better at discerning true from false COVID-19 information.
As displayed in Figure 3, we find that particular characteristics are significantly correlated with discernment of COVID-19 information among our sample of Nigerian Facebook users. Again, by discernment we mean the ability to distinguish true from false headlines, as evidenced by respondents saying they believe the true headlines are accurate and the false ones are not. We give each respondent a discernment score, which is the number of headlines they correctly identified as true or false (out of the 10 they saw) and use this as our outcome variable to see what respondent-level characteristics predict better discernment. We find that reflecting or deliberating, rather than relying on intuition when making decisions, as measured by the Cognitive Reflection Test is significantly associated with truth discernment, as is being older and living in a mostly urban area (Frederick, 2005). 9 The positive correlation we observe between age and discernment is consistent with evidence from survey experiments with U.S. samples (Pennycook et al., 2021), but counter to the analysis of social media data among U.S. users (Grinberg et al., 2019). The positive association we observe between CRT scores and discernment is also consistent with existing research . We also observe important negative correlations with discernment. First, we find that supporting the All Progressives Congress (APC), the current ruling party in Nigeria, is negatively correlated with discernment. We measured partisanship by first asking "do you feel close to any political party?" The 23% of respondents who answered "yes" were then asked, "Which party is that?" Options included All Progressive Congress (APC)-47%, Peoples' Democratic Party (PDP)-48% and other-5%. 10 Second, religiosity is negatively correlated with discernment (p-value = .058). For partisanship, this correlation could be driven by greater trust in the media among ruling party supporters in Nigeria. More religious people, who might be more likely to hold views that God will protect believers from COVID-19, might exert less effort in truth discernment with respect to COVID-19 information because they feel protected. 11 Other studies similarly observe a negative association between religiosity and discernment (Bronstein et al., 2019). Further research is required to understand the mechanism for these correlations.
10 The percent of respondents who do not feel close to any party may seem low, but it is not uncommon in Nigeria. Here partisan affiliation tends to be weaker than say the U.S.-as is evident by the fact that about half (51%) of respondents in the most recent nationally representative Afrobarometer survey said they do not feel close to any political party (Afrobarometer, 2018). Although partisanship is less entrenched in Nigeria, despite being a competitive democracy with two major parties like the U.S., it does not invalidate our finding that APC partisans are worse at discerning true from false headlines than other respondents. 11 Findings from a nationally representative phone survey released by NOI Polls in March 2020 revealed that 26% of Nigerians believed they are immune from COVID-19, and 40% of those people said it was because they are a "child of God." (Source: https://noi-polls.com/covid-19-poll-result-release/) Finally, we find that gender, education, social media use, and voting behavior are not statistically significant predictors of discernment. Recognizing that information about health crises may be particularly emotional, and quite distinct from general online misinformation, the results from this study should not be generalized to other types of misinformation.

Methods
From April 1st to April 3rd, 2020, we conducted an online study of 1,341 Facebook users in Nigeria to understand the relationship between emotional reactions and belief in, reading intentions and sharing intentions of COVID-19-related misinformation. We had three research questions of interest. First, we investigated the relationship between emotions and our outcomes. Specifically, we wanted to know if emotional reactions to headlines were associated with greater belief in, reading, and sharing of false stories (compared to true stories). We selected these outcome measures following existing research on misinformation (Bode & Vraga, 2018;Guess et al., 2019;Tully et al.,2019), and also because they follow the logical process of reacting to a headline-first thinking about whether it is true, deciding whether to click to read it, and then sharing. Second, we examined these relationships for distinct emotions. Finally, we observed the individual characteristics that are associated with the ability to discern true from false headlines.
This research was designed to test the generally assumed but seldom tested claim that emotionally evocative headlines are harder to decipher as true or false. Media outlets have suggested that people fall for spreading misinformation because it is emotionally evocative (Barr, 2019;Lunch, 2019). From psychology research, which suggests that deliberation improves discernment (Bago et al., 2020;, we might also expect emotions to inhibit deliberation and therefore be associated with worse discernment. Drawing on these ideas, we prespecified two main hypotheses for this research. 12 • H1: Respondents will be more likely to believe false headlines when they experience an emotional response. • H2: Respondents will be more likely to want to click to read and share false headlines when they have an emotional response. Using Facebook advertisements, we recruited a sample of social media users in Nigeria aged 18 years old and older. Our sample is 68% male, 32% female, 76% have some university education, with the mean age of respondents being 28 years old. Though quite different from average citizens in Nigeria, this group of social media users-who clicked on an ad to take a survey-is a population of particular interest given the prevalence of misinformation on social media (Allcott et al., 2019;Wang et al., 2019). We focus on Nigeria because it is the most populous country in Africa, a media hub on the continent, and a prominent source of both fact-checking resources and an origin of COVID-19-related misinformation. 13 Facebook users who clicked on our ads were taken to a Qualtrics survey and shown media posts about COVID-19 that had appeared online. Respondents were asked a series of questions about these posts and then answered demographic questions, took the Cognitive Reflection Test (CRT), 14 and reported their social media use. Respondents who completed the survey and provided their phone number were given ~1.50 USD in airtime sent to their mobile phone for their participation.
We obtained the true media posts ("stimuli") from major news outlets in Nigeria, including The Guardian, Daily Post, and Premium Times. The false stimuli came from COVID-19 stories that had appeared online but had subsequently been fact-checked (and verified as untrue) by AfricaCheck.org. All stimuli actually appeared online in Nigeria prior to the experiment. Each respondent saw 10 total COVID-19related headlines, five true and five false, the order of which was random. 15 Headlines were a mix of stories about the state of the pandemic, travel, potential cures, and actions of and statements by political leaders. 16 After viewing each headline, respondents were asked to select any number of six emojis, each representing a distinct basic emotion (Ekman, 1992), to indicate how the headline made them feel (see Figure 4). Before the survey, we conducted a pretest with a separate sample of 76 Nigerians to confirm that the emojis were interpreted correctly. A majority of respondents were able to identify the emotion associated with each emoji. 17 In the main survey, respondents indicated they felt surprise, fear, anger, disgust, happiness, sadness, or any combination of these emotions using these emojis (see Figure 4). The order of the emotions was randomized for each respondent. Respondents also had the option to report that they felt "neutral/no emotion." If they felt any emotion after reading the headline, respondents were asked a follow-up question: "On a scale from 1 (not very strongly) to 7 (very strongly), how strongly did you feel [emotion]?" 18 Since relying on self-reported emotional states requires respondents to accurately understand and report their emotional reactions (Settle 2018), we would have ideally used an automatic or biological measure of emotions immediately after respondents read the headline. Because this was impossible in the context of an online survey, participants were randomly assigned to either give a response to the belief question or the emotion measure first after seeing the headline. We do this because studies show that labeling emotions can act as emotion regulation (Torre & Lieberman, 2018), reducing felt emotions. Asking about emotions after belief, however, is post-treatment with respect to this outcome. We observed no ordering effects in terms of the relationship between reported emotions and belief, and therefore pool data across these two conditions when analyzing the data (see Appendix C).
Belief was measured using the following question: "Do you think this headline accurately describes an event that actually happened?" We also asked whether respondents were (hypothetically) interested 15 The 10 stimuli are presented in Appendix D. 16 We included a mix of political and non-political headlines to reflect the range of the types of headlines respondents would naturally be exposed to on social media concerning the pandemic, based on our own experience following media outlets in Nigeria on Twitter and Facebook at this time. At the end of the survey, respondents were informed which headlines were false and which were true. 17 Accuracy ranged from 55% for the "scared" emoji to 100% for "happy." 18 The results from this measure of emotional strength are consistent with the main results presented here and are provided in Appendix C.
in clicking to read the headline and sharing the headline online. All responses were binary-respondents could either answer "yes" or "no." To analyze the correlation between emotional reactions and our outcomes of interest we use linear mixed-effects models. 19 These models account for the interdependence between observations because outcomes for the same respondent and for the same headline rated by each respondent are likely correlated. We include random effects for respondents and headlines because we are not interested in the specific influence of the particular individual or headline on the outcomes, rather we want to know whether emotions are correlated with the outcomes while controlling for the variation coming from specific headlines and respondents. We run a separate regression for each distinct emotion and interact the emotion dummy with the headline veracity indicator.
Finally, we observe which types of respondents are more likely to believe false, rather than true, COVID-19 information, and assess the correlates of truth discernment among our sample. We use a linear regression to examine whether several demographic variables, as well as CRT scores, self-reported social media use, partisanship, religiosity, and past voting behavior predict discernment-the proportion of the 10 total headlines the respondent correctly identified as true or false (see Figure 3). In line with our preregistered secondary hypothesis, we find that more reflective users, as measured by the Cognitive Reflection Test (CRT) are better at discerning true from false headlines. Figure A2. Percent of respondents who believed false headlines were true.

B.1 Mixed-effects model
For the analysis, we use mixed-effect models with random intercepts for respondents and headlines and a random slope for headlines. 20 This mixed-effects model accounts for the interdependence between observations because participants rate several headlines, and all headlines are assessed by all participants. The model we use in all of our main specifications is: 21 Here represents the emotion rating by respondents, which varies within headlines and between respondents, and is the veracity of the headline (true/false), which varies within respondent and between headlines. The 's estimate the fixed effects, 0 estimates the by-respondent random intercept, and 0 and 1 estimate the by-headline random effects (intercept and slope, respectively). We use linear regression models for all of the main analyses presented in the following tables for each of our outcomes of interest ( ), , , and ℎ . Table B1. Association between neutral (no emotion) and belief, click, and share. 20 We planned to include random slopes for both respondents and headlines but ran into singularity issues. As described in our preanalysis plan, we excluded a random slope from the model, following Brauer and Curtin (2018), so the models would converge. 21 For the "surprise," emotion we only include random intercepts due to singularity.

C.2 Main models with controls
Here we present linear mixed effect models looking at our three outcomes of interest for neutral (no emotion), happiness, and surprise. In each model, we control for gender, age, and education level as we prespecified in our pre-analysis plan. We also control for social media use and support for the governing party (APC), as requested by our anonymous reviewers. The tables reveal a continued positive correlation between neutral (no emotion) and belief discernment (Table C3). In fact, this relationship increases in statistical significance due to improved precision with covariate adjustment. We also observe that the negative association between happiness (Table C4) and surprise (Table C5) with both belief and sharing discernment still holds after including these controls.    Table C6 interacts support for the governing party ("APC") with our main independent variables of interest: headline veracity and an indicator for the absence of any emotional reaction (neutral). We see that the triple interaction term is not statistically significant, but the main interaction of interest (neutral*true) does not change in magnitude. In other words, the relationship between emotional reaction, veracity, and belief in the headline does not differ by partisanship. This is also the case when we include the triple interaction in our regressions for happiness (Table C7). For surprise (Table C8), our main result for belief (surprise*true) remains similar in magnitude as well. Here we also see that the triple interaction term is statistically significant, which means there is a difference between APC supporters' and other respondents' belief in true/false headlines when they are surprised by the headline. More broadly, we also see that APC supporters are more likely to believe, click, and share headlines than nonpartisans and opposition supporters. These results suggest that partisanship is significantly related to belief in headlines-APC supporters are less likely to believe true headlines than other respondents-but partisanship does not alter the relationship between emotions and our outcomes of interest. The relationship between partisanship and reactions to misinformation should be further studied in this context.    Table C9 analyzes the correlation between support for the governing party (APC) and trust in different media sources. Respondents who report supporting the governing party (APC) trust what they see on television and hear on the radio more than other respondents -who either support a different party or do not identify with any political party. Interestingly, however, there is not a statistically significant (at the conventional 5%) difference in trust levels between APC partisans and other respondents with respect to social media. Table C9. Correlation between support for governing party and trust in different media.

C.4 Leave one out analysis
The graphs below plot the coefficients on the interaction terms (emotion*veracity) for the linear mixed effect models predicting belief. These graphs plot estimates on the y axis. The points represent the iterated estimates when 1 out of the 10 headlines is removed from the analysis (to ensure that the results are not being driven by any particular headline). The x-axis indicates which headline is left out. We see that the results are not driven by a particular headline.

C.5 Order of questions
Does being asked about emotions/belief first matter? We check and verify in Table C10 that the order of the questions (emotions and then belief, or belief and then emotions) is not correlated with our outcomes at the conventional 5% level nor is it differentially correlated with discernment for these outcomes.