Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research

Mariam Hejab Alshaibani; Waleed Mugahed Al-Rahmi; Ragad M. Tawafak

doi:10.30935/cedtech/17309

Mariam Hejab Alshaibani ¹, Waleed Mugahed Al-Rahmi ² ^* , Ragad M. Tawafak ³

More Detail

CONT ED TECHNOLOGY, Volume 17, Issue 4, Article No: ep595

https://doi.org/10.30935/cedtech/17309

Submitted: 22 March 2025, Published Online: 20 October 2025

OPEN ACCESS 155 Views 117 Downloads

Download Full Text (PDF)

Abstract

This study introduces and validates the artificial intelligence anxiety scale (AIAS), a novel instrument designed to measure researchers’ anxieties when employing artificial intelligence (AI) tools in academic writing. As AI technologies rapidly infiltrate scholarly work, however, the primary concern grows about their ethical implications, impact on traditional research skills, and the lack of institutional readiness issues that remain underexplored in existing literature. Addressing this critical gap, the AIAS offers a novel framework grounded in real-world academic concerns. Using an inductive approach, data were collected from 219 faculty members and graduate students at Taif University, Saudi Arabia, revealing four core dimensions of AI-related anxiety: (1) concerns about the accuracy of AI outputs, (2) fear of committing plagiarism, (3) lack of institutional guidelines, and (4) fear of losing research skills. Exploratory and confirmatory factor analyses confirmed the existence of these factors, and reliability testing indicated robust internal consistency. By offering the first validated tool specifically tailored to measure AI-related anxiety in academic contexts, this study provides a significant resource for researchers and institutions. Its application can guide universities in devising training and support initiatives to ensure the ethical and practical integration of AI, thereby sustaining the integrity of traditional research competencies and enhancing the overall quality and credibility of academic endeavors.

Keywords: AI anxiety, confirmatory factor analysis, academic research, psychological assessment

References

Acemoğlu, D., & Restrepo, P. (2018). Artificial intelligence, automation, and work. Journal of Economic Perspectives, 33(2), 193–210. https://doi.org/10.3386/w24196
AlFarsi, G., Tawafak, R. M., ElDow, A., Malik, S. I., Jabbar, J., & Al Sideiri, A. (2021). Smart classroom technology in artificial intelligence: A review paper. In Proceedings of the International Conference on Culture Heritage, Education, Sustainable Tourism, and Innovation Technologies (pp. 229–235). https://doi.org/10.5220/0010306502290235
Al-Obaydi, L. H., Rahul, D. R., & Pikhart, M. (2024). The effect of online oral reading on reading comprehension, reading anxiety, and classroom anxiety among EFL learners. Education and Information Technologies, 29(3), 2841–2855. https://doi.org/10.1007/s10639-023-11950-y
Al-Obaydi, L. H., Shakki, F., & Pikhart, M. (2025). How does emergency remote teaching affect EFL students’ writing skills? Cogent Education, 12(1), Article 2500001. https://doi.org/10.1080/2331186X.2025.2500001
Alqahtani, M. A., Alamri, M. M., Sayaf, A. M., & Al-Rahmi, W. M. (2022). Exploring student satisfaction and acceptance of e-learning technologies in Saudi higher education. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.939336
Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, Article 149. https://doi.org/10.3389/fpubh.2018.00149
Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). The Guilford Press.
Chan, C. K. Y. (2023). Is AI changing the rules of academic misconduct? An in-depth look at students’ perceptions of “AI-giarism.” arXiv. https://doi.org/10.48550/arxiv.2306.03358
Cheng, S. M. S., Mok, C. K. P., Leung, Y. W. Y., Ng, S. S., Chan, K. C. K., Ko, F. W., Chen, C., Yiu, K., Lam, B. H. S., Lau, E. H. Y., Chan, K. K. P., Luk, L. L. H., Li, J. K. C., Tsang, L. C. H. Poon, L. L. M., Hui, D. S. C., & Peiris, M. (2022). Neutralizing antibodies against the SARS-CoV-2 Omicron variant BA.1 following homologous and heterologous CoronaVac or BNT162b2 vaccination. Nature Medicine, 28(3), 486–489. https://doi.org/10.1038/s41591-022-01704-7
Chua, S. L., Chen, D.-T., & Wong, A. F. L. (1999). Computer anxiety and its correlates: A meta-analysis. Computers in Human Behavior, 15(5), 609–623. https://doi.org/10.1016/S0747-5632(99)00039-4
Chubb, J., Cowling, P., & Reed, D. (2021). Speeding up to keep up: Exploring the Use of AI in the research process. Springer Nature, 37(4), 1439–1457. https://doi.org/10.1007/s00146-021-01259-0
Clustering, A. S. (2019). International Arab Conference on Information Technology (ACIT). Higher Education, 165, Article 170.
Dahri, N. A., Al-Rahmi, W. M., Almogren, A. S., Yahaya, N., Vighio, M. S., Al-maatuok, Q., Al-Rahmi, A. M., & Al-Adwan, A. S. (2023). Acceptance of mobile learning technology by teachers: Influencing mobile self-efficacy and 21st-century skills-based training. Sustainability, 15(11), Article 8514. https://doi.org/10.3390/su15118514
DeVellis, R. F. (2016). Scale development: Theory and applications (4th ed.). SAGE.
DiStefano, C., Liu, J., Jiang, N., & Shi, D. (2018). Examination of the weighted root mean square residual (WRMR) in evaluating model fit for structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 25(3), 453–466. https://doi.org/10.1080/10705511.2017.1390394
Eldow, A., Shakir, M., Talab, M. A., Muttar, A. K., & Tawafak, R. M. (2006). Literature review of authentication layer for public cloud computing: A meta-analysis. Journal of Theoretical and Applied Information Technology, 97, Article 12.
Evelyn, T. (2021). Surfacing anger and anxiety in graduate research writing. Journal of Multi-Disciplinary Studies, 45(3), 55–70.
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
Habes, M., Jacobson, A. M., Braffett, B. H., Rashid, T., Ryan, C. M., Shou, H., Cui, Y., Davatzikos, C., Luchsinger, J. A., Biessels, G. J., Bebu, I., Gubitosi-Klug, R. A., Bryan, R. N., Nasrallah, I. M., & DCCT/EDIC Research Group. (2023). Patterns of regional brain atrophy and brain aging in middle- and older-aged adults with type 1 diabetes. JAMA Network Open, 6(6), Article e2316182. https://doi.org/10.1001/jamanetworkopen.2023.16182
Hafsa, S., Majid, M. A., & Tawafak, R. M. (2021). Learnability factors of AR usage performance: Validating through survey. In Proceedings of the 2021 International Conference on Software Engineering & Computer Systems and 4^th International Conference on Computational Science and Information Management (pp. 371–376). IEEE. https://doi.org/10.1109/ICSECS52883.2021.00074
Hair Jr, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Cengage Learning.
Hamari, J., Koivisto, J., & Sarsa, H. (2014, January). Does gamification work? -- A literature review of empirical studies on gamification. 2014 47th Hawaii International Conference on System Sciences, 3025–3034. https://doi.org/10.1109/HICSS.2014.377
Hinkin, T. R. (1995). A review of scale development practices in the study of organizations. Journal of Management, 21(5), 967–988. https://doi.org/10.1177/014920639502100509
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
Iqbal Malik, S., Mathew, R., Tawafak, R. M., & Alfarsi, G. (2021). A web-based model to enhance algorithmic thinking for novice programmers. E-Learning and Digital Media, 18(6), 616–633. https://doi.org/10.1177/20427530211026988
Jain, R., & Jain, A. (2023). Generative AI in writing research papers: A new type of algorithmic bias and uncertainty in scholarly work. arXiv. https://doi.org/10.48550/arxiv.2312.10057
Jang, W., & Byon, K. K. (2020). Antecedents and consequence associated with esports gameplay. International Journal of Sports Marketing and Sponsorship, 21(1), 1–22. https://doi.org/10.1108/IJSMS-01-2019-0013
Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). The Guilford Press.
Lecun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539
Lei, P.-W., & Shiverdecker, L. K. (2020). Performance of fit indices in multiple-group confirmatory factor models with categorical outcomes. Structural Equation Modeling: A Multidisciplinary Journal, 27(3), 439–461.
Li, L., Huang, Q., Wang, D. C., Ingbar, D. H., & Wang, X. (2020). Acute lung injury in patients with COVID‐19 infection. Clinical and Translational Medicine, 10(1), 20–27. https://doi.org/10.1002/ctm2.16
Liu, J., & Jagadish, H. V. (2024). Institutional efforts to help academic researchers implement generative AI in research. The MIT Press. https://doi.org/10.1162/99608f92.2c8e7e81
Mohammadkarimi, E. (2023). Teachers’ reflections on academic dishonesty in EFL students’ writings in the era of artificial intelligence. Canadian Philosophy of Education Society, 6(2). https://doi.org/10.37074/jalt.2023.6.2.10
Nilsson, N. J. (2010). The quest for artificial intelligence: A history of ideas and achievements. Cambridge University Press.
Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
Perkins, M., & Roe, J. (2024). Generative AI tools in academic research: Applications and implications for qualitative and quantitative research methodologies. arXiv. https://doi.org/10.48550/arxiv.2408.06872
Perkins, M., Roe, J., Postma, D., McGaughran, J., & Hickerson, D. (2023). Game of tones: Faculty detection of GPT-4 generated content in university assessments. arXiv. https://doi.org/10.48550/arxiv.2305.18081
Pikhart, M., Klimova, B., & Al-Obaydi, L. H. (2024). Exploring university students’ preferences and satisfaction in utilizing digital tools for foreign language learning. Frontiers in Education, 9. https://doi.org/10.3389/feduc.2024.1412377
Rosen, L. D., & Weil, M. M. (1995). Computer anxiety: A cross-cultural comparison of university students in ten countries. Computers in Human Behavior, 11(1), 45–64. https://doi.org/10.1016/0747-5632(94)00021-9
Russell, S. J., & Norvig, P. (2020). Artificial intelligence: A modern approach (4th ed.). Pearson.
Tatnall, A. (2019). Editorial for EAIT issue 2, 2019. Education and Information Technologies, 24, 953–962. https://doi.org/10.1007/s10639-019-09874-7
Tawafak, R. M., Alfarsi, G., AlNuaimi, M. N., Eldow, A., Malik, S. I., & Shakir, M. (2020). Model of faculty experience in e-learning student satisfaction. In Proceedings of the 2020 International Conference on Computer Science and Software Engineering (pp. 83–87). IEEE. https://doi.org/10.1109/CSASE48920.2020.9142071
Tawafak, R. M., Al-Rahmi, W. M., Alshimai, A., Alyoussef, I. Y., & Aldaijy, A. (2025). Exploring the impact of digital gameplay on behavioral intentions in learning graphics programming courses. Contemporary Educational Technology, 17(2), Article ep576. https://doi.org/10.30935/cedtech/16115
Tawafak, R. M., Romli, A., & Arshah, R. A. (2019). E-learning prospect on improving academic performance in Omani universities. IOP Conference Series: Materials Science and Engineering, 551(1), Article 012033. https://doi.org/10.1088/1757-899X/551/1/012033
Teng, M. F. (2023). Scientific writing, reviewing, and editing for open-access TESOL journals: The role of ChatGPT. International Journal of TESOL Studies, 5(1), 87–91. https://doi.org/10.58304/ijts.20230107
Ullah, N., Alnumay, W. S., Al-Rahmi, W. M., Alzahrani, A. I., & Al-Samarraie, H. (2020). Modeling cost saving and innovativeness for blockchain technology adoption by energy management. Energies, 13(18), Article 4783. https://doi.org/10.3390/en13184783
Wang, Y. Y., & Wang, Y. S. (2019). Development and validation of an artificial intelligence anxiety scale: An initial application in predicting motivated learning behavior. Interactive Learning Environments, 30(4), 619–634. https://doi.org/10.1080/10494820.2019.1674887
Wen, B., Xu, C., Wolfe, R., Wang, L. L., & Howe, B. (2024). Mitigating overconfidence in large language models: A behavioral lens on confidence estimation and calibration. In NeurIPS 2024 Workshop on Behavioral Machine Learning.
Yusuf, A., Pervin, N., & González, M. R. (2024). Generative AI and the future of higher education: A threat to academic integrity or reformation? Evidence from multicultural perspectives. Springer Nature, 21, Article 21. https://doi.org/10.1186/s41239-024-00453-6
Zhou, X., Smith, L., & Patel, R. (2023). Exploring ethical and psychological dimensions of AI in academic writing. Technology and Society Review, 12(4), 76–89.

Citation

Alshaibani, M. H., Al-Rahmi, W. M., & Tawafak, R. M. (2025). Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research. Contemporary Educational Technology, 17(4), ep595. https://doi.org/10.30935/cedtech/17309

Alshaibani, M. H., Al-Rahmi, W. M., and Tawafak, R. M. (2025). Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research. Contemporary Educational Technology, 17(4), ep595. https://doi.org/10.30935/cedtech/17309

Alshaibani MH, Al-Rahmi WM, Tawafak RM. Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research. CONT ED TECHNOLOGY. 2025;17(4), ep595. https://doi.org/10.30935/cedtech/17309

Alshaibani, Mariam Hejab, Waleed Mugahed Al-Rahmi, and Ragad M. Tawafak. "Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research". Contemporary Educational Technology 2025 17 no. 4 (2025): ep595. https://doi.org/10.30935/cedtech/17309

Alshaibani, Mariam Hejab et al. "Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research". Contemporary Educational Technology, vol. 17, no. 4, 2025, ep595. https://doi.org/10.30935/cedtech/17309

Alshaibani MH, Al-Rahmi WM, Tawafak RM. Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research. CONT ED TECHNOLOGY. 2025;17(4):ep595. https://doi.org/10.30935/cedtech/17309