Peer Reviewed

Playing Gali Fakta inoculates Indonesian participants against false information

Article Metrics
CrossRef

0

CrossRef Citations

Altmetric Score

59

PDF Downloads

993

Page Views

Although prebunking games have shown promise in Western and English-speaking contexts, there is a notable lack of research on such interventions in countries of the Global South. In response to this gap, we developed Gali Fakta, a new kind of media literacy game specifically tailored for an Indonesian audience. Our findings indicate that participants who engaged with Gali Fakta exhibited significantly greater skepticism toward false news headlines and expressed a reduced likelihood of sharing them. Importantly, playing Gali Fakta did not lead to increased skepticism or decreased sharing intent for factual headlines. These results suggest that Gali Fakta holds promise as a scalable media literacy intervention in Indonesia.

image by geralt on pixabay

Research Questions

  • Can playing Gali Fakta improve participants’ accuracy in assessing false news headlines?
  • Does playing Gali Fakta reduce the intent to share false news headlines?
  • Does playing Gali Fakta lead to an increase in self-reported media literacy?

Research note summary

  • We created a WhatsApp-inspired media literacy game tailored for Indonesian users.
  • We conducted a randomized controlled trial involving N = 1,006 Indonesian participants to assess the impact of playing Gali Fakta. The study aimed to determine if the game could enhance participants’ ability to evaluate the accuracy of false news headlines, reduce their inclination to share false news headlines, and improve self-reported media literacy.
  • Results showed that participants who engaged with Gali Fakta were significantly more inclined to perceive false headlines as less accurate and were less likely to share them. Notably, playing the game did not affect the accuracy or sharing intent regarding factual headlines. However, no improvement was observed in self-reported media literacy scores following gameplay.

Implications

Gali Fakta is a media literacy game inspired by WhatsApp, specifically designed for an Indonesian audience. Its name, “Gali Fakta,” translates to “dig up the facts,” and it was developed through a collaboration between technology experts at Moonshot and researchers at the University of Notre Dame (see Figure 1). The design of the game draws inspiration from the Learn to Discern Program by IREX.

Figure 1. Screenshots from the start of the Gali Fakta game. English translation on the right.

The game is quick to play, taking only about five minutes, and it simulates a group chat with family and friends on a messaging app. Throughout the game, players are tasked with protecting their loved ones from falling for hoaxes. Players can select from three different content areas—health, news, or finance—which prompt people to message them and share various types of content. Each content area provides lessons on safeguarding oneself and others against four types of misinformation: fake social media accounts, confirmation bias, untrustworthy social media sources, and algorithmically-generated filter bubbles.

During gameplay, when players observe their fictional friends or family members sharing misinformation from the mentioned categories, they are presented with two response options. For instance, if a character messages the player about a favorite musical artist requesting money on social media, the player must decide whether the post is factual. In this example, the absence of a verification symbol indicates the post is false. If the player answers incorrectly, they receive a prompt explaining the correct response (see Figure 2). Throughout the game, players engage in conversations with fictitious family members, offering ten different responses and opportunities to learn about misinformation techniques. Before its release, the game script underwent review and revision by Indonesian subject matter experts.

Figure 2. Screenshots from Gali Fakta when a participant correctly identifies misinformation in the game along with the lesson afterward. English translation on the right.

Gali Fakta aims to create psychological inoculation by first providing a forewarning that the user’s friends and family may be spreading misinformation and then exposing the user to misinformation shared by fictitious friends and family members in the simulated WhatsApp environment along with lessons on how to detect such misinformation. We hypothesized that after playing Gali Fakta participants would become more conscious of their social media behavior and more discerning when encountering false information. The false information in our study was a variety of false headlines, some of which were politically charged and sensationalist. Others were simply factually inaccurate (see Appendix B for details). Our study revealed that playing Gali Fakta was associated with higher accuracy in evaluating false news headlines and a decreased self-reported intention to share false headlines.

Media literacy, prebunking, and Indonesia

Indonesia ranks as the world’s fourth-largest country, with a rapidly increasing number of citizens gaining digital access. Currently, around 77% of the population is online, and this figure is projected to reach 90% by 2025 (Kemp, 2023; Nurhayati-Wolff, 2023). This growing online presence also creates a vulnerability to online misinformation. Researchers have found that misinformation was evident during the 2019 Indonesian general election, particularly on social media platforms (Theisen et al., 2021). Besides political misinformation, misinformation related to health and finance also circulates widely in the Indonesian information ecosystem (McRae et al., 2022; Mujani & Kuipers 2020; Nasir & Nurmansyah, 2020). Research has also indicated that students in Indonesia face challenges in distinguishing between false and factual news (Syam & Nurrahmi, 2020). A 2020 report by GeoPoll found that over 40% of Indonesians admitted they only read the headline and “rarely” or “never” read the article before sharing it on social media (McDonnell & MacKinnon, 2020). More research is needed to determine the most effective media literacy tools for different countries as well as different types of misinformation. Notably, misinformation and disinformation disproportionately affect individuals with lower levels of media literacy (Jalli & Idris, 2019).

To combat this issue, prebunking games based on inoculation theory have shown promise in helping individuals discern fact from fiction and reducing their inclination to share misinformation (Basol et al., 2020). Inoculation theory involves building resistance to misinformation by exposing individuals to small doses of false or misleading information in a controlled setting (Compton et al., 2021). However, these prebunking games have limitations, such as potentially increasing skepticism towards both factual and false information (Modirrousta-Galian & Higham, 2022). Additionally, individuals may still share false information due to social or political motives, despite possessing higher levels of media literacy (Sirlin et al., 2021). Moreover, most prebunking games and media literacy interventions have been tailored to Western, English-speaking audiences and may not be culturally relevant to other populations. Iyengar, Gupta, & Priya (2022) found that participants from India increased their ability to detect misinformation after playing the Western-based media literacy intervention Bad News Game. However, the authors note that their English-speaking and highly educated sample was not representative of much of the population of India. Harjani and colleagues (2022) created a media literacy game that was designed for participants in India and tested its effectiveness with a sample of participants from north India. Unfortunately, they did not find that their game improved participants’ ability to evaluate misinformation or reduced willingness to share misinformation with others. The authors note that future work may benefit by partnering with local researchers and universities in order to better adapt interventions originally designed with a Western audience in mind. They also note the importance of considering the potential lack of digital literacy of a rural audience, which could limit the effectiveness of playing an online media literacy game.

Recognizing the popularity of WhatsApp in Indonesia and its susceptibility to misinformation due to the ease of forwarding messages without fact-checking, we designed Gali Fakta to mimic the WhatsApp interface (Banaji et al., 2019; de Freitas Melo et al., 2020; Neyazi et al., 2022; Yustitia & Asharianto, 2020). Political groups have also exploited WhatsApp during the 2019 Indonesian election to disseminate misleading or false information through political memes (Baulch et al., 2022). Our aim was to develop a game that aligns with the information environment of its players to enhance prebunking effectiveness. We partnered with Indonesian scholars and media experts during the development of the game to constantly improve the feel, aesthetics, and clarity of our lessons. For example, there are laws against spreading false information in Indonesia (Lamb & Teresia, 2022) and neighboring Malaysia (Sipalan, 2021). Therefore, we aimed to avoid any negative associations with learning about misinformation through a game that might unintentionally glorify misinformation spreading. Instead, Gali Fakta focuses on empowering users to protect their friends and family from hoaxes. We conducted an evaluation to assess whether playing Gali Fakta could enhance the ability of Indonesian participants to identify false headlines, reduce their likelihood of sharing false headlines, and improve their self-reported media literacy habits.

Findings

Finding 1: Playing Gali Fakta significantly increased participants’ skepticism for false news headlines but not for factual headlines.

We conducted a series of ordinary least squares (OLS) regressions to investigate whether playing Gali Fakta predicted participants’ tendency to rate false news headlines as more inaccurate, indicating increased skepticism. OLS regressions identify relationships between variables by minimizing the sum of squared errors.

Figure 3. Coefficient plot illustrating the effects of Gali Fakta on both factual and false headlines. Participants significantly increased their skepticism for false headlines but not for factual headlines. (*p < .05, **p < .01, ***p < .001).

Participants in the Gali Fakta condition significantly rated false news headlines as more inaccurate compared to those in the control condition while controlling for age, gender, education, media literacy, urban versus rural residence, religiosity, income, and political ideology (p < .001). Notably, engaging with Gali Fakta did not lead to a significant increase in inaccuracy ratings of factual headlines (p = .173). Thus, playing Gali Fakta effectively enhanced participants’ discernment of false headlines, but this heightened skepticism did not extend to factual headlines. Full regression tables, means, and effect sizes are provided in Appendix A.

Finding 2: Playing Gali Fakta significantly reduced participants’ intent to share false headlines but not factual headlines.

In the subsequent analysis, we conducted a series of OLS regressions to assess whether playing Gali Fakta was associated with a decreased intent to share false headlines. Figures 3 and 4 illustrate the effects of playing Gali Fakta on evaluating both factual and false headlines.

Figure 4. Coefficient plot illustrating the effects of Gali Fakta on both factual and false headlines. Participants in the Gali Fakta condition significantly reduced their intent to share false headlines but not factual headlines (*p < .05, ** p < .01, ***p < .001).

Results revealed that engaging with Gali Fakta significantly predicted a reduced intent to share false headlines even when controlling for age, gender, education, media literacy, urban vs rural residence, religiosity, income, and political ideology (p = .002). The regression coefficients for playing Gali Fakta were 0.192 for rating false headlines and 0.215 for sharing false headlines. This means that participants who played our brief media literacy game rated false headlines as more inaccurate by about 0.2 of a point on a 1–5 scale point and decreased their likelihood to share by about 0.2 on a 1–5 scale. Notably, playing Gali Fakta did not lead to a significant decrease in sharing intent for factual headlines (p = .114). Once again, these findings underscore the effectiveness of our media literacy game in fostering skepticism and caution regarding misinformation, while not affecting the sharing intent for accurate information.

We also analyzed to what extent participants differentiated between factual and false headlines.  As a robustness check, we analyzed discernment (i.e., the difference in ratings between factual and false headlines) and still found that playing Gali Fakta significantly improved both headline accuracy discernment and sharing discernment (see Appendix A). While our results demonstrate that playing Gali Faktaimproves participants’ ability to discern factual headlines versus false news headlines, it is unclear if playing Gali Faktawould impact social media behavior or override social and political motives to share misinformation. We do find it encouraging that these positive effects hold while controlling for education, media literacy, and political ideology.

Finding 3: Playing Gali Fakta does not impact self-reported media literacy.

In addition to assessing headline ratings and sharing intention, we also examined whether playing Gali Fakta would lead to an increase in self-reported media literacy. Our analysis did not reveal a significant increase in participants’ self-reported media literacy following engagement with Gali Fakta (p = .622). This null result may be attributed to limitations inherent in self-report media literacy scales, as they may not comprehensively measure individuals’ actual media literacy skills (Jones-Jang et al., 2021). Alternatively, it is possible that Gali Fakta may not prompt participants to reflect on the specific types of questions posed in the media literacy scale used.

Future studies could explore more targeted inquiries, such as assessing participants’ confidence in correcting misinformation shared by others online, which aligns more closely with the tasks undertaken in Gali Fakta. While playing Gali Fakta did not influence self-reported media literacy scores, we found that participants who played the game versus a control group were significantly better able to evaluate the accuracy of false news headlines and also reported a reduced likelihood of sharing false news headlines. Future work can also investigate the duration of this effect along with whether Gali Fakta can be successful in different country-level populations after translations. Finally, we measured the ability to evaluate false text-based headlines as our measure of misinformation. However, this is only one type of misinformation, and it does not capture the ability to evaluate more complex types of misinformation, such as hyperpartisan misinformation and misleading information from mainstream news that is not completely false. Future research could develop media literacy interventions that are specifically designed to teach individuals to identify these more complicated types of misinformation.

While our effect sizes are small, we believe these positive results are encouraging from playing a brief media literacy game. Gali Fakta is an extremely scalable intervention, and small effects could have more meaningful cumulative effects if the game is shared widely in public campaigns and in educational settings. Large social media accounts can simply share Gali Fakta with their audience and encourage their followers to play the game and share it with their friends and family. Additionally, Gali Fakta offers a fun and simple exercise to add to media literacy curriculums. Gali Fakta is freely available in both its original Bahasa Indonesian version1https://literata.id/game/ as well as the translated English version.2https://galifakta.com/

Methods

Participants

We recruited 1,006 Indonesian participants through Bilendi & Respondi’s survey platform, with 495 assigned to the Gali Fakta condition and 511 to the control condition (see Appendix A for power analysis).  Participants from Bilendi & Respondi’s database have enough digital literacy to sign up for an online survey platform, which may yield an important baseline of digital literacy that is required to understand and learn from our game. Data was collected in February of 2023. We excluded 191 participants who did not respond to the question about political ideology, aiming to control for politics given the political nature of some headlines. The average age of participants was 38.53 (SD = 11.43), with 52.89% male and 81.51% residing in urban areas. On a religiosity scale ranging from 1 (not religious at all) to 5 (very religious), the average score was 3.80 (SD = 0.81). The average level of conservatism, rated on a scale from 1 (very liberal) to 5 (very conservative), was 3.37 (SD = 1.01). Regarding education, 74.85% completed tertiary school, and 99.7% completed secondary school. The average annual income was 133,401,566.79 Indonesian Rupiah or 8,357 USD (SD = 544,834,772.12 for IDR and 34,133.90 for USD).

Procedure

Participants were randomly assigned to play either Gali Fakta or Tetris (see Figure 5), a game chosen as an effective control condition due to its comparable time and cognitive demands (Roozenbeek & van der Linden, 2020). After playing the assigned game, participants evaluated seven factual and seven false headlines covering topics of climate change (two headlines), COVID-19 (three headlines), and non-political matters (two headlines), such as money-saving techniques (see Appendix B for headline list).

Figure 5. Illustrated experimental design of the present study.

The topics were equal across both the factual and false headlines. Headlines were chosen from actual headlines circulating on Indonesian social media during February 2023 that provided a mix of political topics (climate change and COVID-19) as well as nonpolitical topics. In addition to being politically charged topics, false information about climate change and COVID-19 was spread by disinformation campaigns throughout Indonesia (Kurnia et al., 2024; Muzykant et al., 2021). Disinformation surrounding these topics also represents attempts to discredit scientific expertise. Participants rated the accuracy of each headline and their likelihood of sharing it on a 1–5 Likert scale. Additionally, participants completed an 11-item media literacy scale from Austin et al. (2021) counterbalanced with the headline questions. This scale assessed media literacy regarding news source credibility and critical evaluation of news content (see Appendix B). Finally, participants provided demographic information, including age, gender, education, geographic location, religiosity, income, and political ideology.

Topics
Download PDF
Cite this Essay

Facciani, M. J., Apriliawati, D., & Weninger, T. (2024). Playing Gali Fakta inoculates Indonesian participants against false information. Harvard Kennedy School (HKS) Misinformation Review. https://doi.org/10.37016/mr-2020-152

Links

Bibliography

Austin, E. W., Austin, B. W., Willoughby, J. F., Amram, O., & Domgaard, S. (2021). How media literacy and science media literacy predicted the adoption of protective behaviors amidst the COVID-19 pandemic. Journal of Health Communication, 26(4), 239–252. https://doi.org/10.1080/10810730.2021.1899345

Banaji, S., Bhat, R., Agarwal, A., Passanha, N., & Sadhana Pravin, M. (2019). WhatsApp vigilantes: An exploration of citizen reception and circulation of WhatsApp misinformation linked to mob violence in India. London School of Economics and Political Science. http://eprints.lse.ac.uk/id/eprint/104316

Basol, M., Roozenbeek, J., & van der Linden, S. (2020). Good news about bad news: Gamified inoculation boosts confidence and cognitive immunity against fake news. Journal of Cognition, 3(1). https://doi.org/10.5334%2Fjoc.91

Baulch, E., Matamoros-Fernández, A., & Suwana, F. (2022). Memetic persuasion and WhatsAppification in Indonesia’s 2019 presidential election. New Media & Society, 26(5), 2473–2491. https://doi.org/10.1177/14614448221088274

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.

Compton, J., van der Linden, S., Cook, J., & Basol, M. (2021). Inoculation theory in the post‐truth era: Extant findings and new frontiers for contested science, misinformation, and conspiracy theories. Social and Personality Psychology Compass, 15(6), e12602. https://doi.org/10.1111/spc3.12602

de Freitas Melo, P., Vieira, C. C., Garimella, K., de Melo, P. O. V., & Benevenuto, F. (2020). Can WhatsApp counter misinformation by limiting message forwarding? In H. Chefifi, S. Gaito, J. F. Mendes, E. Moro, & L. M. Rocha (Eds.), Complex networks and their applications VIII: Volume 1 proceedings of the eighth international conference on complex networks and their applications COMPLEX NETWORKS 2019 (pp. 372–384). Springer International Publishing. https://doi.org/10.1007/978-3-030-36687-2_31

Faul, F., Erdfelder, E., Buchner, A., & Lang, A.G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149–1160.  https://doi.org/10.3758/BRM.41.4.1149

Harjani, T., Basol, M. S., Roozenbeek, J., & van der Linden, S. (2023). Gamified inoculation against misinformation in India: A randomized control trial. Journal of Trial and Error3(1), 14–56,  https://doi.org/10.36850/e12

Iyengar, A., Gupta, P., & Priya, N. (2023). Inoculation against conspiracy theories: A consumer side approach to India’s fake news problem. Applied Cognitive Psychology37(2), 290–303. https://doi.org/10.1002/acp.3995

Jalli, N., & Idris, I. (2019). Fake news and elections in two Southeast Asian nations: A comparative study of Malaysia general election 2018 and Indonesia presidential election 2019. In A. Hamid, Y. P. Setianto, Riswanda, A. Fuad, & M.D. Hikmawan (Eds.), International conference on democratisation in Southeast Asia (ICDeSA 2019) (pp. 138–148). Atlantis Press. https://doi.org/10.2991/icdesa-19.2019.30

Jones-Jang, S. M., Mortensen, T., & Liu, J. (2021). Does media literacy help identification of fake news? Information literacy helps, but other literacies don’t. American Behavioral Scientist, 65(2), 371–388. https://doi.org/10.1177/0002764219869406

Kemp, S. (2023, February 9). Digital 2023 Indonesia. Datareportal. https://datareportal.com/reports/digital-2023-indonesia

Kurnia, N., Putri, T., Kusumaningtyas, A., Abisono, F., Putrawidjoyo, A., Sasmita Jati-Karim, M., & Socarana, B. (2024). Tackling climate crisis misinformation in Indonesia.  https://research.rug.nl/en/publications/tackling-climate-crisis-misinformation-in-indonesia

Lamb, K. & Teresia, A. (2022, December 9). Indonesia’s new laws a threat to privacy, press and human rights, says UN. Reuters. https://www.reuters.com/world/asia-pacific/indonesias-new-laws-threat-privacy-press-human-rights-says-un-2022-12-09/

McRae, D., del Mar Quiroga, M., Russo-Batterham, D., Doyle, K., & Platform, A. (2022). A pro-government disinformation campaign on Indonesian Papua. Harvard Kennedy School (HKS) Misinformation Review, 3(5). https://doi.org/10.37016/mr-2020-108

McDonnell, I., & MacKinnon, T. (2020). Case study: Misinformation in Indonesia. GeoPoll. https://www.geopoll.com/misinformation-indonesia/

Modirrousta-Galian, A., & Higham, P. A. (2023). Gamified inoculation interventions do not improve discrimination between true and fake news: Reanalyzing existing research with receiver operating characteristic analysis. Journal of Experimental Psychology: General, 152(9), 2411–2437. https://doi.org/10.1037/xge0001395

Mujani, S., & Kuipers, N. (2020). Who believed misinformation during the 2019 Indonesian election? Asian Survey, 60(6), 1029–1043,  https://doi.org/10.1525/as.2020.60.6.1029

Muzykant, V. L., Muqsith, M. A., Pratomo, R. R., & Barabash, V. (2021). Fake news on COVID-19 in Indonesia. In D. M. Berube (Ed.), Pandemic communication and resilience (pp. 363-378). Springer International Publishing. https://doi.org/10.1007/978-3-030-77344-1_10

Nasir, N. M., & Nurmansyah, M. I. (2020). Misinformation related to COVID-19 in Indonesia. Institutional Repository UIN Syarif Hidayatullah. https://repository.uinjkt.ac.id/dspace/handle/123456789/63519

Neyazi, T. A., Yi Kai Ng, A., Kuru, O., & Muhtadi, B. (2022). Who gets exposed to political misinformation in a hybrid media environment? The case of the 2019 Indonesian election. Social Media + Society, 8(3). https://doi.org/10.1177/20563051221122792

Nurhayati-Wolff, H.  (2023). Number of internet users in Indonesia 20172028. Statista. https://www.statista.com/statistics/254456/number-of-internet-users-in-indonesia/

Roozenbeek, J., & van der Linden, S. (2020). Breaking Harmony Square: A game that “inoculates” against political misinformation. Harvard Kennedy School (HKS) Misinformation Review, 1(8). https://doi.org/10.37016/mr-2020-47

Sipalan, P. (2021, March 12). Malaysia defends coronavirus fake news law amid outcry. Reuters. https://www.reuters.com/business/media-telecom/malaysia-defends-coronavirus-fake-news-law-amid-outcry-2021-03-12/

Sirlin, N., Epstein, Z., Arechar, A. A., & Rand, D. G. (2021). Digital literacy is associated with more discerning accuracy judgments but not sharing intentions. Harvard Kennedy School (HKS) Misinformation Review, 2(6). https://doi.org/10.37016/mr-2020-83

Syam, H. M., & Nurrahmi, F. (2020). “I don’t know if it is fake or real news”: How little Indonesian University students understand social media literacy. Jurnal Komunikasi: Malaysian Journal of Communication, 36(2), 92–105. https://doi.org/10.17576/JKMJC-2020-3602-06

Theisen, W., Brogan, J., Thomas, P. B., Moreira, D., Phoa, P., Weninger, T., & Scheirer, W. (2021). Automatic discovery of political meme genres with diverse appearances. In C. Budak, M. Cha, D. Quercia, & L. Xie (Eds.), Proceedings of the International AAAI Conference on Web and Social Media, 15(1), 714–726). https://doi.org/10.1609/icwsm.v15i1.18097

Yustitia, S., & Asharianto, P. D. (2020). Misinformation and disinformation of COVID-19 on social media in Indonesia. Proceeding of LPPM UPN “Veteran” Yogyakarta Conference Series 2020 – Engineering and Sciences (ESS), 1(1), 51–65. https://proceeding.researchsynergypress.com/index.php/pss/article/view/180

Funding

Portions of this work were supported by the U.S. Agency for International Development (USAID) Cooperative Agreement Number 7200AA18CA00059.

Competing Interests

The authors declare no competing interests.

Ethics

Data collection methods were evaluated and approved by the Institutional Review Board (IRB) of The University of Notre Dame. All subjects verbally consented to participation in this survey and to receive follow-up SMS messages to their mobile phone.

Copyright

This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided that the original author and source are properly credited.

Data Availability

All materials needed to replicate this study are available via the Harvard Dataverse: https://doi.org/10.7910/DVN/BKXRNQ

Acknowledgements

We would like to thank USAID, Moonshot, Brave Factor, and IREX for their help during this project.