Research note: Likes, sarcasm and politics: Youth responses to a platform-initiated media literacy campaign on social media

---

Implications

Amidst an alarming rise in online misinformation (McGinty & Gyenes, 2020; Pasquetto et al., 2020), there has been a growing interest in media literacy education as a way to combat the rapid spread of misinformation (Bulger & Davison, 2018; Chang et al., 2020; Roozenbeek & Van Der Linden, 2020; Tully et al., 2020). However, the focus of both research and practice has been on the integration of media literacy instruction into young people’s various educational experiences, rather than implementing and evaluating such initiatives in situ, on social media (Bulger & Davison, 2018; Pasquetto et al., 2020)—which is known to be their central news source (Common Sense Media, 2019). Indeed, often due to the lack of necessary data (see Pasquetto et al., 2020), there has been insufficient attention devoted to assessing media literacy initiatives on social media; furthermore, existing studies focus on older platforms like Facebook or Twitter (e.g., Guess et al., 2019; Vraga & Tully, 2021), and we lack a nuanced understanding of these dynamics on the platforms most popular with youth today. Given this context, our study aimed to facilitate a bottom-up and naturalistic study of user responses to a media literacy education campaign on TikTok, an increasingly popular youth-oriented social media platform.

Based on a large-scale qualitative analysis of user responses to this media literacy education campaign, we find that user reactions were mixed, and highly political in nature. On the positive side, our findings show that young people appreciate the urgency of media literacy education and understand its relevance to social media and to their own online habits; this attitude can be seen as an opening for future initiatives and a solid foundation to build off of. While critical and sarcastic comments abounded, this is to be expected on social media in general (Muresan et al., 2016; Zappavigna, 2012), and even more so when it comes to educational topics (Literat, 2021). Significantly, the more negative responses to this campaign can serve as rich feedback for the design of future media literacy initiatives for youth, particularly on social media. Below, we tease out a few of these directions. 

Key takeaways and recommendations

First, it is clear that youth were keenly aware of TikTok’s “rap sheet” in regard to curbing misinformation (or not), and that this perception—shaped by both micro-level and macro-level factors (e.g., personal experience with reporting/moderation on the platform, mass media reports, political rhetoric like Trump’s anti-TikTok discourse)—had a strong impact on their attitude towards the campaign. Commenters in this study openly rebuked TikTok for sharing educational materials addressing fake news while failing to curb the spread of misinformation on their platform, and this perceived hypocrisy generated a negative response to the campaign. This sense of discontent with content moderation on TikTok resulted in a diminished level of trust, which negatively affected user engagement with the campaign videos. As Brunk and colleagues (2019) found, providing explicit reasoning around content moderation practices promotes a higher level of user trust, and might thus alleviate any communicative hindrances to learning. Therefore, in order to ensure the effectiveness of such educational initiatives, we recommend that campaign creators address any perceived inconsistencies between action and rhetoric. In the case of platform-run initiatives, this might translate to increased transparency in communicating how decisions are made regarding user reports of misinformation or content violations. 

Much of the negative feedback to the campaign can be attributed to the fact that it was perceived—in both promotion and design/content—as being at odds with the ethos and function of TikTok in youth lives. Youth engage on social networks such as TikTok with specific expectations related to both the aesthetic and social nature of their participation on the platform and are quick to sniff out content they perceive as inauthentic (boyd, 2014). While TikTok’s campaign attempted to meet these expectations (e.g., by using humor and leveraging TikTok microcelebrities to act out these video skits), users did not connect with the language and style employed in the videos. This might be understood in relation to the shift in TikTok users’ aesthetic appetite, which is moving away from “visual theatrics” and towards more relatable content that taps into the current trends and latest memes (Abidin, 2021). A key way to preempt this pitfall and facilitate a sense of relevance and nowness is to include the main stakeholders, youth themselves, in the design of such initiatives. Participatory design allows participants to enact their identities, providing valuable insight into the characteristics, perceptions, and expectations of the target population (Coenraad et al., 2019); significantly, in the media literacy sphere, it has been shown to be an effective strategy for the design of news literacy initiatives for youth (see Literat et al., 2020).

Regarding campaign promotion and dissemination, we found that youth were extremely frustrated with how the videos were being pushed, often repeatedly, onto their feeds. This essentially begs the question of how—or whether—platforms can promote such educational content without it being negatively perceived or dismissed for its top-down, forceful, unsolicited approach? In line with our previous recommendation, and as a way to facilitate a more organic reach, we recommend that platforms consider utilizing more participatory, community-oriented channels. This could take the form, for instance, of initiating participatory activities (e.g., TikTok challenges), thereby distributing the ownership of the campaign and encouraging more user buy-in, and/or engaging influencers in the promotion of content. Indeed, recent initiatives leveraging TikTok influencers in the promotion of educational campaigns, including around COVID-19 messaging, highlight “the efficacy of influencers in information dissemination and control in the online space” (Abidin et al., 2021, p. 124).

Another key takeaway from this research is the highly political nature of user responses to this campaign. Certainly, this political focus was also influenced by the timing of the videos, given the prominence of the Black Lives Matter movement and the presidential election in this timeframe. However, it is a powerful illustration of the politicized nature of media literacy education in the current sociocultural landscape. Designers of future media literacy initiatives—on social media and elsewhere—need to keep in mind that, in this landscape, even apolitical content might be politicized and framed in polarizing terms by users. More broadly, these findings also reinforce the link between media literacy and civic education (Mihailidis, 2018; Mihailidis & Thevenin, 2013), and the fact that, particularly in this political and technological context, each is a prerequisite for the other.

Evidence

Positive responses: Understanding and embracing the educational aims of the videos

On the positive side, users expressed appreciation for the aims of this campaign, and for TikTok taking this initiative. Many users seemed to understand and appreciate the educational aims of the videos, and complimented the videos for being “helpful,” “cool,” “educational,” “timely,” “engaging,” and “necessary:”

“ok but this tip is awesome because it encourages critical thinking” 

“basically telling you to do your own research! that’s refreshing to hear from a social media platform”

Significantly, there was evidence of self-reflection and perceived usefulness among these responses:

“wow you’re right, my media has always been left leaning, i see all my favorite celebrities pushing a left leaning agenda, maybe that’s a scary bias!” 

“Me ready to send this to my transphobic family“

Users also understood the urgency and significance of media literacy education more largely. They often thanked TikTok for taking this initiative, sometimes as part of a positive comparison with other social media platforms:

“Thank you for doing this ! People spreading around false information has been getting way out of hand”

“Yeahhhhhhh this is so necessary. Thank you tik tok for doing this.”

“this a very important topic thank you tiktok for doing more than most social media cites” 

“Facebook should learn a thing or two from you. I deleted my FB account”  

Other comments implied that these videos on TikTok were just as effective or even more effective than the media literacy instruction they received in school:

“Ok but im in college and why am I finally understanding this😂” 

“Who needs school when TikTok teaches us something new every day” 

“This short video was more effective than my mandatory 5 hour “unconscious bias” training 🙄” 

“Why can’t my history teachers give me this instead of an Intro To Trusted Sourcing lesson”  

Negative responses: Confusion, annoyance, disconnection and perceived hypocrisy  

However, negative comments—often sarcastic in tone—far outweighed the more positive takes above. Indeed, the corpus included about twice as many negative comments than positive ones—though, as mentioned above, this balance should be taken with a grain of salt, given the nature of social media expression, compounded by the educational focus of the initiative. These negative reactions were grounded in: confusion about the aim of the videos; annoyance at videos being pushed on users’ “for you page” (FYP);¹TikTok’s “For You Page” is an algorithmically curated feed that functions as the platform’s homepage. disliking the video content and aesthetics; and perceived hypocrisy due to TikTok’s own actions and policies. 

Users were confused as to why they appeared on their FYP, as this kind of content did not seem to match their interests and regular FYP content. Some thought the videos were ads; interestingly, this pattern was most prominent on the videos about distinguishing facts from opinions and, respectively, about questioning graphics, as both of these videos used humorous ads for fake products as a way to anchor their narratives. 

A large number of users expressed their annoyance at being unable to block these videos, and being forced to watch them multiple times in a short period of time:

“This is 4 times In a row I see these adds”  

“i swear if u put this on my fyp 9272836292539352935936 times-”  

“I swear if I see one more video from this acc I’ll just delete tiktok all together-💀” 

Others disliked the approach that the videos took, for the following reasons: 1) use of outdated language and aesthetics (e.g., “are these targeted at millennials because I’m getting 90s nostalgia”; “What kind of early 2000s commercial is this”); 2) videos were perceived as too didactic, and incongruent with the perceived function of TikTok in youth lives (e.g., “This is TIKTOK not school so bye”; “ok i’m on my phone during class so i don’t have to do class and yet tik tok is showing me how to identify the source”); 3) the educational content was seen as too basic (e.g., “I learned this is 3rd grade”; “Why is this here? 😳 Kindergarten lesson”); and 4) videos were perceived as too boring (e.g., “I- I just wanna know what level of bored”; “There’s no way that was only a minute I feel like I just sat through orientation😅”). 

Interestingly, some users were resistant to this media literacy campaign just based on their existing perceptions of TikTok’s actions regarding media literacy. In particular, they called out TikTok’s failure to control the spread of misinformation on its platform; here, many users brought up stories of reporting questionable content, with TikTok taking no action about it. Trust in the effectiveness of the reporting feature and the enforcement of community guidelines seemed low: 

“Yes, except reporting TikToks does no good because y’all pick and choose what guidelines to withhold.”

“i reported a video of a dude throwing stuff at homeless people and laughing and they said they found nothing wrong with it”  

“[The reporting] feature is functionally worthless. Doesn’t matter what you report – it goes through a bot and gets spat right back out” 

“Also, when you report false or misleading information nothing happens because it doesn’t violate their community guidelines.” 

Along the same lines, others brought up TikTok’s practice of over-/mis-censoring or shadowbanning users, an accusation that was often politicized along partisan or racial lines (e.g., “I’m more worried about tiktok continuing to silence black creators and uplift white creators who are appropriating black culture…”; “and why is BLM [Black Lives Matter] showdown ban [shadowbanned]”²See also Rosenblatt (2021) for the wider context of race-based shadowbanning and content suppression on TikTok.). Finally, users also referenced TikTok’s business practices and connection to China—a government associated with heavy media censorship and misinformation. The latter perception was undoubtedly shaped by Donald Trump’s recent discourse and efforts to ban TikTok. 

The political nature of responses

A key finding from this research study was that—whether positive, negative, or neutral—a large proportion of comments (roughly a third of the data corpus) were political, i.e., making reference to political actors, groups, or issues. Political comments were posted on all videos, but were especially prominent on the video about distinguishing facts from opinions, which proved to be a very politicized issue. Key topics of discussion included Black Lives Matter, partisanship, the presidential election, as well as feminist, LGBTQ, racial discrimination issues. Many commenters explicitly expressed a particular political identity (Democrat / Republican, pro/anti-Trump, liberal/conservative, left/right, etc.). Comments often included political slogans (e.g., “Trump 2020”, “BIDEN 2020”) and were not afraid to express political stances or challenging others’ views. A prevalent pattern was mentioning the opposing party or ideology in comments and saying that they need to watch these videos—thus implying that the other party/ideology is lacking media literacy skills: 

“@ all trump supporters should see this” 

“Liberals take notes” 

“Democrats seeing this 👩‍🦯👩‍🦯👩‍🦯👩‍🦯👩‍🦯”

“people who watch fox news needs this“

“the left ain’t gonna like this one”

Significantly, the rationale behind the videos was politicized too, framing media literacy education as an inherently political endeavor:

“Bias education? This is political.”  

“Are you making this so you can silence Trump supporters like every other social media platform?? This is a load of bullshit”

“This is about trump right-“

Given the prominence of political comments, some users expressed annoyance with how often politics was brought up in responses to these videos: 

“take the politics elsewhere 😀👍”  

“No need to make it political” 

“I literally went to the comments and counted to 5 knowing I would find someone that brings up politics before I finished counting”

At the same time, political comments usually got more responses (e.g., likes, replies) than non-political ones, and exchanges were longer and more engaged, though often impolite and uncivil. 

Methods

This study aimed to analyze how users respond to a platform-initiated media literacy education campaign on youth-oriented social media, with a focus on the lessons that we might draw from these responses in terms of targeted media literacy education initiatives for youth. While there is broad societal consensus about the urgency and significance of media literacy education, especially for young people, we still lack a grounded, bottom-up understanding of youth attitudes towards such initiatives. Moreover, trying to capture these attitudes through self-report methods like surveys or interviews runs the risk of producing biased results, due to social desirability effects (especially strong when it comes to the topic of media literacy and misinformation; see Tully et al., 2020), as well as the normative pressures that shape the way youth might respond to adult questions about their educational experiences. Here, using qualitative content analysis on a large corpus of youth reactions to a media literacy campaign on social media facilitates a naturalistic study of young people’s attitudes towards such initiatives in situ, “in the wild.”

Research site 

Our research site, TikTok, is a social media platform where users can create, share and browse short looping videos. Users can edit their videos by adding music or soundtracks, visual effects, and filters. They can follow other users, like and comment on videos, and communicate via direct messages. Users can discover content by searching for specific tags, users or soundtracks, but are also exposed to a custom feed of recommended videos on their For You Page. 

The popularity of TikTok has grown significantly since its parent company, ByteDance, acquired its predecessor, musical.ly, in late 2018. In 2020, TikTok reached 45.4 million users in the United States (eMarketer, 2020a), and the audience growth rate is predicted to reach 15.2% in 2021 (eMarketer, 2020b). The app is particularly popular with youth, who make up 60% of its demographic in the U.S. (App Ape, 2020). The cultural impact of TikTok on youth lives is significant, as the platform shapes youth culture, crowns grassroots celebrities, and popularizes new music, dances, and cultural trends (Herrman, 2020; The Youth Lab, 2019). In the United States, TikTok has been a key venue for youth political expression (see Herrman, 2020) and has drawn the ire of former President Donald Trump, who unsuccessfully tried to ban it in 2020.

Data corpus

This study relied on the qualitative content analysis of 11,449 public comments posted in response to five videos published by an official TikTok account (specific handle hidden for anonymization purposes). Created by TikTok US in partnership with external experts, the five videos were part of a strategic media literacy campaign designed to highlight—via the use of humorous narratives acted out by TikTok microcelebrities—skills like fact-checking, assessing the credibility of information and visuals, understanding bias, and distinguishing facts from opinions. The videos were very similar in terms of their comedic approach and aesthetics, making for a congruent campaign. They were extremely popular on the platform, accumulating a total of 53.8m views, 1.63m likes, and 11k shares; at the same time, it is worth noting that the high view count was likely boosted by their being disseminated from an official TikTok account and pushed onto users’ For You Pages.³In view of the data available and our methodological approach, we cannot ascertain the specific ages of users represented in this corpus of comments. However, based on the age demographics of TikTok and the content of the comments themselves (e.g., references to school, use of youth lingo, references to youth culture), we can infer that the vast majority of commenters are youth. Similarly, while the exact geographic location of commenters is unknown, the vast majority were likely from the United States, in view of the aims and distribution of the campaign (i.e., produced by TikTok US and targeting American TikTok users) as well as the content of comments (making reference to US politics, US education, etc.).

Data analysis 

The data corpus was analyzed qualitatively, using a thematic analysis approach (Braun & Clarke, 2006) in order to identify patterns and nuance in the data. Four researchers (authors 2-5 of this study) participated in the qualitative coding process. The first analytical step involved identifying comments that were related to the content and/or educational aims of the videos, and were thus relevant to understanding youth responses to this media literacy initiative. These comments were labeled with simple codes (e.g., “positive,” “confused,” “sarcastic,”⁴In coding for sarcasm, we maintained a bottom-up, grounded approach that considered both the content of each comment and its larger context of expression (including relationship to the video, to other user replies, and to relevant real-life events). We paid attention to both the text of the comment, and any emojis or hashtags that might encourage a sarcastic reading. At the same time, we acknowledge the difficulties in objectively and definitively detecting sarcasm online (see, e.g., Muresan et al., 2015) and the inescapable subjectivity inherent in this process. “comparison with school,” “comparison with other social media,” “political,” etc.) that facilitated a birds-eye understanding of the data corpus. Then, in a second round of analysis, we read the comments again and identified subthemes in the data; finally, these subthemes were further synthesized and coalesced into the key findings presented above.