Journal of Teaching Innovation and Reform

Articles

Vol. 1 (2025)

Empirical Validation of the CHAT-RV Framework: AI-Driven Hoax Filtering and Reference Validation among Indonesian Undergraduates

Published
2025-10-17

Abstract

This study investigates the effectiveness of the CHAT-RV (ChatGPT for Hoax Analysis and Truthful Reference Validation) framework among 100 Indonesian undergraduates drawn from five academic disciplines (Islamic Education, Natural Sciences, Mathematics, Guidance and Counseling, and English Studies). Employing a quantitative survey design, data were collected using a structured Likert-scale instrument assessing four dimensions of the CHAT-RV model: hoax recognition, citation validation, epistemic trust calibration, and ethical AI usage. Results demonstrate significant improvements in students’ epistemic literacy, with Islamic Education and English majors outperforming peers in hoax recognition and citation triangulation. Factor analysis confirmed the reliability of the four-dimensional structure (Cronbach’s α = .87), while regression results indicated that citation validation (β = .31, p < .01) and ethical AI awareness (β = .28, p < .01) were the strongest predictors of digital literacy outcomes.

References

  1. Adarkwah, M. A. (2025). GenAI-Infused Adult Learning in the Digital Era: A Conceptual Framework for Higher Education. Adult Learning, 36(3), 149-161. https://doi.org/10.1177/10451595241271161
  2. Almulla, M. A. (2020). The Effectiveness of the Project-Based Learning (PBL) Approach as a Way to Engage Students in Learning. SAGE Open, 10(3), 2158244020938702. https://doi.org/10.1177/2158244020938702
  3. Biswas, S. (2023). ChatGPT and the Future of Medical Writing. Radiology, 307(2), e223312. https://doi.org/10.1148/radiol.223312
  4. Bridges, L. M., McElroy, K., & Welhouse, Z. (2024). Generative Artificial Intelligence: 8 Critical Questions for Libraries. Journal of Library Administration, 64(1), 66-79. https://doi.org/10.1080/01930826.2024.2292484
  5. Cacicio, S., & Riggs, R. (2023). ChatGPT: Leveraging AI to Support Personalized Teaching and Learning. Adult Literacy Education: The International Journal of Literacy, Language, and Numeracy, 5(2), 70-74. https://doi.org/10.35847/SCacicio.RRiggs.5.2.70
  6. Chiu, T. K. F. (2024). The impact of Generative AI (GenAI) on practices, policies and research direction in education: A case of ChatGPT and Midjourney. Interactive Learning Environments, 32(10), 6187-6203. https://doi.org/10.1080/10494820.2023.2253861
  7. Ciampa, K., Wolfe, Z. M., & Bronstein, B. (2023a). ChatGPT in education: Transforming digital literacy practices. Journal of Adolescent & Adult Literacy, 67(3), 186-195. https://doi.org/10.1002/jaal.1310
  8. Ciampa, K., Wolfe, Z. M., & Bronstein, B. (2023b). ChatGPT in education: Transforming digital literacy practices. Journal of Adolescent & Adult Literacy, 67(3), 186-195. https://doi.org/10.1002/jaal.1310
  9. Creswell, J. W., & Creswell, J. D. (2022). Research Design. SAGE Publications, Inc. https://us.sagepub.com/en-us/nam/research-design/book270550
  10. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334. https://doi.org/10.1007/BF02310555
  11. Davison, R. M., Chughtai, H., Nielsen, P., Marabelli, M., Iannacci, F., van Offenbeek, M., Tarafdar, M., Trenz, M., Techatassanasoontorn, A. A., Díaz Andrade, A., & Panteli, N. (2024). The ethics of using generative AI for qualitative data analysis. Information Systems Journal, 34(5), 1433-1439. https://doi.org/10.1111/isj.12504
  12. Dekov, I. (2025, June 25). The Psychology of Influence: How Social Media Algorithms Fuel Manipulation and Echo Chambers. VISION_FACTORY. https://www.visionfactory.org/post/the-psychology-of-influence-how-social-media-algorithms-fuel-manipulation-and-echo-chambers
  13. Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., … Wright, R. (2023). “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71. Scopus. https://doi.org/10.1016/j.ijinfomgt.2023.102642
  14. Ecker, P. A. (2025). Algorithms, Polarization, and the Digital Age: A Literature Review. In P. A. Ecker (Ed.), The Digital Reinforcement of Bias and Belief: Understanding the Cognitive and Social Impact of Web-Based Information Processing (pp. 21-43). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-89998-0_2
  15. Floridi, L. (2023). The Ethics of Artificial Intelligence: Principles, Challenges, and Opportunities. Oxford University Press. https://doi.org/10.1093/oso/9780198883098.001.0001
  16. González-Pérez, L. I., Montoya, M. S. R., & García-Peñalvo, F. J. (2022). Habilitadores tecnológicos 4.0 para impulsar la educación abierta: Aportaciones para las recomendaciones de la UNESCO. RIED-Revista Iberoamericana de Educación a Distancia, 25(2), 23-48. https://doi.org/10.5944/ried.25.2.33088
  17. Hamed, A. A., Zachara-Szymanska, M., & Wu, X. (2024). Safeguarding authenticity for mitigating the harms of generative AI: Issues, research agenda, and policies for detection, fact-checking, and ethical AI. iScience, 27(2). https://doi.org/10.1016/j.isci.2024.108782
  18. Hayashi, K., & Yuan, K. H. (2023). On the Relationship Between Coefficient Alpha and Closeness Between Factors and Principal Components for the Multi-factor Model. Springer Proc. Math. Stat., 422, 173-185. https://doi.org/10.1007/978-3-031-27781-8_16
  19. Knowles, M. S., III, E. F. H., & Swanson, R. A. (2014). The Adult Learner: The definitive classic in adult education and human resource development (8th ed.). Routledge.
  20. Kreps, S., & Kriner, D. (2023). How AI Threatens Democracy. Journal of Democracy, 34(4), 122-131. https://doi.org/10.1353/jod.2023.a907693
  21. Kurt, D. S. (2020, January 8). Problem-Based Learning (PBL). Educational Technology. https://educationaltechnology.net/problem-based-learning-pbl/
  22. Lund, B. D., & Wang, T. (2023). Chatting about ChatGPT: How may AI and GPT impact academia and libraries? Library Hi Tech News, 40(3), 26-29. https://doi.org/10.1108/LHTN-01-2023-0009
  23. Mishra, P., Warr, M., & Islam, R. (2023). TPACK in the age of ChatGPT and Generative AI. Journal of Digital Learning in Teacher Education, 39(4), 235-251. https://doi.org/10.1080/21532974.2023.2247480
  24. Nurhayati, S., Taufikin, T., Judijanto, L., & Musa, S. (2025). Towards Effective Artificial Intelligence-Driven Learning in Indonesian Child Education: Understanding Parental Readiness, Challenges, and Policy Implications. Educational Process: International Journal. https://www.edupij.com/index/arsiv/76/526/towards-effective-artificial-intelligence-driven-learning-in-indonesian-child-education-understanding-parental-readiness-challenges-and-policy-implications. https://doi.org/10.22521/edupij.2025.15.155
  25. Salaverría, R., & Cardoso, G. (2023). Future of disinformation studies: Emerging research fields. Profesional de La Información, 32(5), Article 5. https://doi.org/10.3145/epi.2023.sep.25
  26. Shah, S. B., Thapa, S., Acharya, A., Rauniyar, K., Poudel, S., Jain, S., Masood, A., & Naseem, U. (2024a). Navigating the Web of Disinformation and Misinformation: Large Language Models as Double-Edged Swords. IEEE Access, 1-1. https://doi.org/10.1109/ACCESS.2024.3406644
  27. Shah, S. B., Thapa, S., Acharya, A., Rauniyar, K., Poudel, S., Jain, S., Masood, A., & Naseem, U. (2024b). Navigating the Web of Disinformation and Misinformation: Large Language Models as Double-Edged Swords. IEEE Access, 1-1. https://doi.org/10.1109/ACCESS.2024.3406644
  28. Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017). Fake News Detection on Social Media: A Data Mining Perspective (No. arXiv:1708.01967). arXiv. https://doi.org/10.1145/3137597.3137600
  29. Stark, L. (2023). Breaking Up (with) AI Ethics. American Literature, 95(2), 365-379. https://doi.org/10.1215/00029831-10575148
  30. Taufikin, T., Judijanto, L., & Nurhayati, S. (2025). Enhancing Undergraduate Research Writing Using ChatGPT: Effectiveness, Student Perceptions, and Ethical Implications. Jurnal Edutech Undiksha, 13(1), 148-157. https://doi.org/10.23887/jeu.v13i1.94305
  31. Thorp, H. H. (2024). ChatGPT to the rescue? Science, 385(6714), 1143-1143.https://doi.org/10.1126/science.adt0007
  32. Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), 15. https://doi.org/10.1186/s40561-023-00237-x
  33. UNESCO. (2023). Guidance for generative AI in education and research. United Nations Educational, Scientific and Cultural Organization. https://unesdoc.unesco.org/ark:/48223/pf0000386693
  34. Wineburg, S., & McGrew, S. (2019). Lateral Reading and the Nature of Expertise: Reading Less and Learning More When Evaluating Digital Information. Teachers College Record, 121(11), 1-40. https://doi.org/10.1177/016146811912101102
  35. Zhou, S., & Yang, S. (2024). The Advantages of Introducing Multimedia Technology into College Students’ Physical Education. Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, 584 LNICST, 61-71. Scopus. https://doi.org/10.1007/978-3-031-63142-9_6