Empirical Validation of the CHAT-RV Framework: AI-Driven Hoax Filtering and Reference Validation among Indonesian Undergraduates
DOI:
https://doi.org/10.65638/2978-5634.2025.01.07Keywords:
Generative AI, CHAT-RV framework, Hoax filtering, Reference validation, Digital epistemic literacyAbstract
This study investigates the effectiveness of the CHAT-RV (ChatGPT for Hoax Analysis and Truthful Reference Validation) framework among 100 Indonesian undergraduates drawn from five academic disciplines (Islamic Education, Natural Sciences, Mathematics, Guidance and Counseling, and English Studies). Employing a quantitative survey design, data were collected using a structured Likert-scale instrument assessing four dimensions of the CHAT-RV model: hoax recognition, citation validation, epistemic trust calibration, and ethical AI usage. Results demonstrate significant improvements in students’ epistemic literacy, with Islamic Education and English majors outperforming peers in hoax recognition and citation triangulation. Factor analysis confirmed the reliability of the four-dimensional structure (Cronbach’s α = .87), while regression results indicated that citation validation (β = .31, p < .01) and ethical AI awareness (β = .28, p < .01) were the strongest predictors of digital literacy outcomes.
References
Adarkwah, M. A. (2025). GenAI-Infused Adult Learning in the Digital Era: A Conceptual Framework for Higher Education. Adult Learning, 36(3), 149-161. https://doi.org/10.1177/10451595241271161
Almulla, M. A. (2020). The Effectiveness of the Project-Based Learning (PBL) Approach as a Way to Engage Students in Learning. SAGE Open, 10(3), 2158244020938702. https://doi.org/10.1177/2158244020938702
Biswas, S. (2023). ChatGPT and the Future of Medical Writing. Radiology, 307(2), e223312. https://doi.org/10.1148/radiol.223312
Bridges, L. M., McElroy, K., & Welhouse, Z. (2024). Generative Artificial Intelligence: 8 Critical Questions for Libraries. Journal of Library Administration, 64(1), 66-79. https://doi.org/10.1080/01930826.2024.2292484
Cacicio, S., & Riggs, R. (2023). ChatGPT: Leveraging AI to Support Personalized Teaching and Learning. Adult Literacy Education: The International Journal of Literacy, Language, and Numeracy, 5(2), 70-74. https://doi.org/10.35847/SCacicio.RRiggs.5.2.70
Chiu, T. K. F. (2024). The impact of Generative AI (GenAI) on practices, policies and research direction in education: A case of ChatGPT and Midjourney. Interactive Learning Environments, 32(10), 6187-6203. https://doi.org/10.1080/10494820.2023.2253861
Ciampa, K., Wolfe, Z. M., & Bronstein, B. (2023a). ChatGPT in education: Transforming digital literacy practices. Journal of Adolescent & Adult Literacy, 67(3), 186-195. https://doi.org/10.1002/jaal.1310
Ciampa, K., Wolfe, Z. M., & Bronstein, B. (2023b). ChatGPT in education: Transforming digital literacy practices. Journal of Adolescent & Adult Literacy, 67(3), 186-195. https://doi.org/10.1002/jaal.1310
Creswell, J. W., & Creswell, J. D. (2022). Research Design. SAGE Publications, Inc. https://us.sagepub.com/en-us/nam/research-design/book270550
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334. https://doi.org/10.1007/BF02310555
Davison, R. M., Chughtai, H., Nielsen, P., Marabelli, M., Iannacci, F., van Offenbeek, M., Tarafdar, M., Trenz, M., Techatassanasoontorn, A. A., Díaz Andrade, A., & Panteli, N. (2024). The ethics of using generative AI for qualitative data analysis. Information Systems Journal, 34(5), 1433-1439. https://doi.org/10.1111/isj.12504
Dekov, I. (2025, June 25). The Psychology of Influence: How Social Media Algorithms Fuel Manipulation and Echo Chambers. VISION_FACTORY. https://www.visionfactory.org/post/the-psychology-of-influence-how-social-media-algorithms-fuel-manipulation-and-echo-chambers
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., … Wright, R. (2023). “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71. Scopus. https://doi.org/10.1016/j.ijinfomgt.2023.102642
Ecker, P. A. (2025). Algorithms, Polarization, and the Digital Age: A Literature Review. In P. A. Ecker (Ed.), The Digital Reinforcement of Bias and Belief: Understanding the Cognitive and Social Impact of Web-Based Information Processing (pp. 21-43). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-89998-0_2
Floridi, L. (2023). The Ethics of Artificial Intelligence: Principles, Challenges, and Opportunities. Oxford University Press. https://doi.org/10.1093/oso/9780198883098.001.0001
González-Pérez, L. I., Montoya, M. S. R., & García-Peñalvo, F. J. (2022). Habilitadores tecnológicos 4.0 para impulsar la educación abierta: Aportaciones para las recomendaciones de la UNESCO. RIED-Revista Iberoamericana de Educación a Distancia, 25(2), 23-48. https://doi.org/10.5944/ried.25.2.33088
Hamed, A. A., Zachara-Szymanska, M., & Wu, X. (2024). Safeguarding authenticity for mitigating the harms of generative AI: Issues, research agenda, and policies for detection, fact-checking, and ethical AI. iScience, 27(2). https://doi.org/10.1016/j.isci.2024.108782
Hayashi, K., & Yuan, K. H. (2023). On the Relationship Between Coefficient Alpha and Closeness Between Factors and Principal Components for the Multi-factor Model. Springer Proc. Math. Stat., 422, 173-185. https://doi.org/10.1007/978-3-031-27781-8_16
Knowles, M. S., III, E. F. H., & Swanson, R. A. (2014). The Adult Learner: The definitive classic in adult education and human resource development (8th ed.). Routledge.
Kreps, S., & Kriner, D. (2023). How AI Threatens Democracy. Journal of Democracy, 34(4), 122-131. https://doi.org/10.1353/jod.2023.a907693
Kurt, D. S. (2020, January 8). Problem-Based Learning (PBL). Educational Technology. https://educationaltechnology.net/problem-based-learning-pbl/
Lund, B. D., & Wang, T. (2023). Chatting about ChatGPT: How may AI and GPT impact academia and libraries? Library Hi Tech News, 40(3), 26-29. https://doi.org/10.1108/LHTN-01-2023-0009
Mishra, P., Warr, M., & Islam, R. (2023). TPACK in the age of ChatGPT and Generative AI. Journal of Digital Learning in Teacher Education, 39(4), 235-251. https://doi.org/10.1080/21532974.2023.2247480
Nurhayati, S., Taufikin, T., Judijanto, L., & Musa, S. (2025). Towards Effective Artificial Intelligence-Driven Learning in Indonesian Child Education: Understanding Parental Readiness, Challenges, and Policy Implications. Educational Process: International Journal. https://www.edupij.com/index/arsiv/76/526/towards-effective-artificial-intelligence-driven-learning-in-indonesian-child-education-understanding-parental-readiness-challenges-and-policy-implications. https://doi.org/10.22521/edupij.2025.15.155
Salaverría, R., & Cardoso, G. (2023). Future of disinformation studies: Emerging research fields. Profesional de La Información, 32(5), Article 5. https://doi.org/10.3145/epi.2023.sep.25
Shah, S. B., Thapa, S., Acharya, A., Rauniyar, K., Poudel, S., Jain, S., Masood, A., & Naseem, U. (2024a). Navigating the Web of Disinformation and Misinformation: Large Language Models as Double-Edged Swords. IEEE Access, 1-1. https://doi.org/10.1109/ACCESS.2024.3406644
Shah, S. B., Thapa, S., Acharya, A., Rauniyar, K., Poudel, S., Jain, S., Masood, A., & Naseem, U. (2024b). Navigating the Web of Disinformation and Misinformation: Large Language Models as Double-Edged Swords. IEEE Access, 1-1. https://doi.org/10.1109/ACCESS.2024.3406644
Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017). Fake News Detection on Social Media: A Data Mining Perspective (No. arXiv:1708.01967). arXiv. https://doi.org/10.1145/3137597.3137600
Stark, L. (2023). Breaking Up (with) AI Ethics. American Literature, 95(2), 365-379. https://doi.org/10.1215/00029831-10575148
Taufikin, T., Judijanto, L., & Nurhayati, S. (2025). Enhancing Undergraduate Research Writing Using ChatGPT: Effectiveness, Student Perceptions, and Ethical Implications. Jurnal Edutech Undiksha, 13(1), 148-157. https://doi.org/10.23887/jeu.v13i1.94305
Thorp, H. H. (2024). ChatGPT to the rescue? Science, 385(6714), 1143-1143.https://doi.org/10.1126/science.adt0007
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), 15. https://doi.org/10.1186/s40561-023-00237-x
UNESCO. (2023). Guidance for generative AI in education and research. United Nations Educational, Scientific and Cultural Organization. https://unesdoc.unesco.org/ark:/48223/pf0000386693
Wineburg, S., & McGrew, S. (2019). Lateral Reading and the Nature of Expertise: Reading Less and Learning More When Evaluating Digital Information. Teachers College Record, 121(11), 1-40. https://doi.org/10.1177/016146811912101102
Zhou, S., & Yang, S. (2024). The Advantages of Introducing Multimedia Technology into College Students’ Physical Education. Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, 584 LNICST, 61-71. Scopus. https://doi.org/10.1007/978-3-031-63142-9_6
