AI and critical thinking: A mixed-method exploration of students’ uses of Ask.SMILE for question-generating practices

Trang Phan; Mariya A. Yukhymenko-Lescroart; Curtis J. Bonk

doi:10.30935/cedtech/18936

Trang Phan ¹ ^* , Mariya A. Yukhymenko-Lescroart ¹, Curtis J. Bonk ²

More Detail

CONT ED TECHNOLOGY, Volume 18, Issue 3, Article No: ep672

https://doi.org/10.30935/cedtech/18936

Submitted: 05 November 2025, Published Online: 07 July 2026

OPEN ACCESS 24 Views 15 Downloads

Download Full Text (PDF)

Abstract

The research study aimed to explore how preservice teachers engaged with an artificial intelligence (AI)-based tool called Ask.SMILE to develop higher-order level of questions based on the criteria of Bloom’s taxonomy. Specifically, we evaluated students’ questioning techniques, motivation, satisfaction with Ask.SMILE, and assessed their aspirations to use the tool in future classrooms. Using a mixed-methods approach, 178 participants generated over 50 questions each, received feedback from Ask.SMILE, and reflected on their experiences through written responses. The results revealed that Ask.SMILE supported participants in creating questions aligned with Bloom’s taxonomy principles, encouraging iterative improvement and fostering metacognitive awareness. Participants reported enhanced understanding of inquiry-based learning and recognized the importance of using structured questioning strategies to promote critical thinking. While the findings highlight the educational potential of AI tools like Ask.SMILE, limitations include reliance on self-reported data, the absence of longitudinal tracking, and varying levels of AI proficiency among participants. It is suggested that future research should focus on the long-term impact of AI-supported questioning strategies and practices in diverse classroom contexts, and professional development to enhance teacher proficiency with AI tools.

Keywords: question-generating, techniques, strategies, inquiry-based learning, engagement

References

AACTE. (2022). Colleges of education: A national portrait. American Association of Colleges for Teacher Education. https://aacte.org/wp-content/uploads/2022/03/Colleges-of-Education-A-National-Portrait-Executive-Summary.pdf
Aflalo, E. (2021). Students generating questions as a way of learning. Active Learning in Higher Education, 22(1), 63-75. https://doi.org/10.1177/1469787418769120
Aguilar, O. G., Mortimer, E. F., & Scott, P. (2010). Learning from and responding to students’ questions: The authoritative and dialogic tension. Journal of Research in Science Teaching, 47, 174-193. https://doi.org/10.1002/tea.20315
Ali, F., Choy, D., Divaharan, S., Tay, H., & Chen, W. (2023). Supporting self-directed learning and self-assessment using TeacherGAIA, a generative AI chatbot application: Learning approaches and prompt engineering. Learning: Research and Practice, 9(2), 135-147. https://doi.org/10.1080/23735082.2023.2258886
Anderson L, W., & Krathwohl D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Pearson Longman.
Anistyasari, Y., Ekohariadi, E., Yani, M. T., Wiriawan, O., & Hidayati, S. C. (2024). An artificial intelligence-based tool for student-generated questions. E3S Web of Conferences, 513, Article 040018. https://doi.org/10.1051/e3sconf/202451302003
Becker, R. R. (2000). The critical role of students’ questions in literacy development. The Educational Forum, 64(3), 261-271. https://doi.org/10.1080/00131720008984763
Bozkurt, A. (2023). Generative artificial intelligence (AI) powered conversational educational agents: The inevitable paradigm shift. Asian Journal of Distance Education, 18(1), 198-204.
Buckner, E., & Kim, P. (2014). Integrating technology and pedagogy for inquiry-based learning: The Stanford mobile inquiry-based learning environment (SMILE). Prospects, 44(1), 99-118. https://doi.org/10.1007/s11125-013-9269-7
Chin, C., & Brown, D. E. (2002). Student generated questions: A meaningful aspect of learning in science. International Journal a/Science Education, 24(5), 521-549. https://doi.org/10.1080/09500690110095249
Chin, C., & Osborne, J. (2008). Students’ questions: A potential resource for teaching and learning science. Studies in Science Education, 44(1), 1-39. https://doi.org/10.1080/03057260701828101
Chiu, T. K.F., Xia, Q., Zhou, X., Chai, C. S., & Cheng, M. (2023). Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Computers and Education: Artificial Intelligence, 4, Article 100118. https://doi.org/10.1016/j.caeai.2022.100118
Crompton, H., & Burke, D. (2024). The educational affordances and challenges of ChatGPT: State of the field. TechTrends, 68, 380-392. https://doi.org/10.1007/s11528-024-00939-0
Davey, B., & McBride, S. (1986). Effects of question-generation training on reading comprehension. Journal of Educational Psychology, 78(4), 256-262. https://doi.org/10.1037/0022-0663.78.4.256
Digital Education Council. (2024). AI or not AI: What students want. Digital Education Council global AI student survey 2024. Digital Education Council. https://www.digitaleducationcouncil.com/post/digital-education-council-global-ai-student-survey-2024
Ennis, R. H. (1985). The logical basis for measuring CT skills. Educational Leadership, 43(2), 44-48.
Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In J. B. Baron, & R. J. Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9-26). W.H. Freeman.
Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction: Executive summary, The Delphi report. California Academic Press.
Holbeck, R. (2025). Helping students develop AI prompting skills for critical thinking. Faculty Focus. https://www.facultyfocus.com/articles/teaching-with-technology-articles/helping-students-develop-ai-prompting-skills-for-critical-thinking/
Hou, X., Chang, Z., Lu, Z., Wenli, C., Wan, L., Feng, W., Hu, H., & Guo, Q. (2025). EduThink4AI: Bridging educational critical thinking and multi-agent LLM Systems. arXiv.
Hsu, H. Y., & Kim, P. (2016). Preservice teachers’ uses of SMILE to enact student-generated questioning practices. International Journal of Innovation in Education, 3(2-3), 110-121. https://doi.org/10.1504/IJIIE.2016.081547
IBM Corp. (2023). IBM SPSS statistics for Macintosh, version 29.0.1.0. IBM Corp. https://www.ibm.com/support/pages/downloading-ibm-spss-statistics-2901
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., ... Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, Article 102274. https://doi.org/10.1016/j.lindif.2023.102274
Khasawneh, E., Hodge-Zickerman, A., York, C. S., Smith, T. J., & Mayall, H. (2023). Examining the effect of inquiry-based learning versus traditional lecture-based learning on students’ achievement in college algebra. International Electronic Journal of Mathematics Education, 18(1), Article em0724. https://doi.org/10.29333/iejme/12715
Kim, M., & Adlof, L. (2024). Adapting to the future: ChatGPT as a means for supporting constructivist learning environments. TechTrends, 68, 37-46 (2024). https://doi.org/10.1007/s11528-023-00899-x
Kim, P. (2024). Trends of AI in education and new competencies for the future at the exploring the ethical use of AI TIP Conference [Video]. YouTube. https://youtu.be/DsUEhxlNku8?si=h2vXfm-P2zNfdg5o
Kim, P., & An, J. Y. (2016). New evaluation vector through the Stanford mobile inquiry-based learning environment (SMILE) for participatory action research. Healthcare Informatics Research, 22(3), 164-171. https://doi.org/10.4258/hir.2016.22.3.164
Kim, P., & Lee, J. (2020). Evolution of online learning environments and the emergence of intelligent MOOCs. In K. Zhang, C. J. Bonk, T. C. Reeves, & T. H. Reynolds (Eds.), MOOCs and open education in the Global South: Challenges, successes, and opportunities (pp. 329-341). Routledge.
Kim, P., Miranda, T., & Olaciregui, C. (2008). Pocket school: Exploring mobile technology as a sustainable literacy education option for underserved indigenous children in Latin America. International Journal of Educational Development, 28(4), 435-445. https://doi.org/10.1016/j.ijedudev.2007.11.002
Kim, P., Wang, W., & Bonk, C. J. (2025). Generative AI as a coach to help students enhance proficiency in question formulation. Journal of Educational Computing Research, 63(3), 565-586. https://doi.org/10.1177/07356331251314222
King, A. (1997). ASK to THINK-TEL WHY: A model of transactive peer tutoring for scaffolding higher level complex learning. Educational Psychologist, 32(4), 221-235. https://doi.org/10.1207/s15326985ep3204_3
Li, Z., Wang, C., & Bonk, C. J. (2024). Exploring the utility of ChatGPT for self-directed online language learning. Online Learning, 28(3), 157-180. https://doi.org/10.24059/olj.v28i3.4497
McGivney, E., & Kim, P. (2016). Using technology to teach the art of asking questions. Stanford Social Innovation Review. https://doi.org/10.48558/89AT-W123
Mollick, E. (2024). Co-intelligence: Living and working with AI. Penguin.
Palincsar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction, 1(2), 117-175. https://doi.org/10.1207/s1532690xci0102_1
Paul, R., & Elder, L. (2006). Critical thinking: Tools for taking charge of your learning and your life. Pearson Prentice Hall.
Phan, T., & Zhu, M. (2026). Asking better questions with AI: Exploring student inquiry, critical thinking growth, and the interpretive role of Ask.SMILE. TechTrends. https://doi.org/10.1007/s11528-026-01192-3
Phan, T., Zheng, X., & Zhu, M. (2026). Comparative analysis of AI-generated and peer feedback on student-generated questions in teacher education. AI Enhanced Learning, 2(1), 99-121. https://doi.org/10.70725/493067ulyywm
Quintana, C., Reiser, B., Davis, E. A., Krajcik, J., Golan, R., Kyza, E., Edelson, D., & Soloway, E. (2002). Evolving a scaffolding design framework for designing educational software. In P. Bell, R. Stevens, & T. Satwicz (Eds.), Proceedings of the 5^th International Conference for the Learning Sciences. Erlbaum.
Rogoff, B. (1990). Apprenticeship in thinking: Cognitive development in a social context. Oxford Academic. https://doi.org/10.1093/oso/9780195059731.001.0001
Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning & Teaching, 6(1), 342-363. https://doi.org/10.37074/jalt.2023.6.1.9
Schroeder, R. (2024). The AI augmented professor of 2024. Inside Higher Ed. https://www.insidehighered.com/opinion/blogs/online-trending-now/2024/05/08/ai-augmented-professor-2024
Seeds of Empowerment. (2025). SMILE LLM project brief: Phi-2 model deployment on Raspberry Pi 5 using llama.cpp. Seeds of Empowerment. https://www.seedsofempowerment.org/smile-llm/
Shi, Z., Castellucci, G., Filice, S., Kuzi, S., Kravi, E., Agichtein, E., Rokhlenko, O., & Malmasi, S. (2025). Ambiguity detection and uncertainty calibration for question answering with large language models. In Proceedings of the 5^th Workshop on Trustworthy NLP (pp. 41-55). ACL Anthology. https://doi.org/10.18653/v1/2025.trustnlp-main.4
Singer, J., Marx, R. W., Krajcik, J., & Chambers, J. C. (2000). Constructing extended inquiry projects: Curriculum materials for science education reform. Educational Psychologist, 35(3), 165-178. https://doi.org/10.1207/S15326985EP3503_3
Terada, Y. (2020). Using student-generated questions to promote deeper thinking. Edutopia.
United States Census Bureau. (2025). Back to school: August 2025. United States Census Bureau. https://www.census.gov/newsroom/stories/back-to-school.html
Vendrell, M., & Johnston, S. K. (2026). Scaffolding critical thinking with generative AI: Design principles for integrating large language models in higher education. Computers and Education: Artificial Intelligence, 10, Article 100572. https://doi.org/10.1016/j.caeai.2026.100572
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.
Wang, C., Li, Z., & Bonk, C. J. (2024). Understanding self-directed learning in AI-assisted writing: A mixed methods study of postsecondary learners. Computers & Education: Artificial Intelligence, 6, Article 100247. https://doi.org/10.1016/j.caeai.2024.100247
Wertsch, J. V. (1991). Voices of the mind: A sociocultural approach to mediated action. Harvard University Press.
Wu, L., Liu, Y., How, M. L., & He, S. (2023). Investigating student-generated questioning in a technology-enabled elementary science classroom: A case study. Education Sciences, 13(2), Article 158. https://doi.org/10.3390/educsci13020158
Yu, F. Y., & Wu, W. S. (2020). Effects of student-generated feedback corresponding to answers to online student-generated questions on learning: What, why, and how? Computers & Education, 145, Article 103. https://doi.org/10.1016/j.compedu.2019.103723
Yu, F. Y., Chang, Y. L., & Wu, H. L. (2015). The effects of an online student question-generation strategy on elementary school student English learning. Research and Practice in Technology Enhanced Learning, 10, Article 24. https://doi.org/10.1186/s41039-015-0023-z
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – Where are the educators? International Journal of Educational Technology in Higher Education, 16(1), Article 39. https://doi.org/10.1186/s41239-019-0171-0
Zhang, K., Bonk, C. J., Reeves, T. C., & Reynolds, T. H. (Eds.). (2020). MOOCs and open education in the Global South: Challenges, successes, and opportunities. Routledge. https://doi.org/10.4324/9780429398919
Zhi, J., Kumar, H., & Lee, M. (2026). Investigating the effects of LLM use on critical thinking under time constraints: Access timing and time availability. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (pp. 1-21). ACM. https://doi.org/10.1145/3772318.3791796

Citation

Phan, T., Yukhymenko-Lescroart, M. A., & Bonk, C. J. (2026). AI and critical thinking: A mixed-method exploration of students’ uses of Ask.SMILE for question-generating practices. Contemporary Educational Technology, 18(3), Article ep672. https://doi.org/10.30935/cedtech/18936

Phan, T., Yukhymenko-Lescroart, M. A., and Bonk, C. J. (2026). AI and critical thinking: A mixed-method exploration of students’ uses of Ask.SMILE for question-generating practices. Contemporary Educational Technology, 18(3), ep672. https://doi.org/10.30935/cedtech/18936

Phan T, Yukhymenko-Lescroart MA, Bonk CJ. AI and critical thinking: A mixed-method exploration of students’ uses of Ask.SMILE for question-generating practices. CONT ED TECHNOLOGY. 2026;18(3), ep672. https://doi.org/10.30935/cedtech/18936

Phan, Trang, Mariya A. Yukhymenko-Lescroart, and Curtis J. Bonk. "AI and critical thinking: A mixed-method exploration of students’ uses of Ask.SMILE for question-generating practices". Contemporary Educational Technology 2026 18 no. 3 (2026): ep672. https://doi.org/10.30935/cedtech/18936

Phan, Trang et al. "AI and critical thinking: A mixed-method exploration of students’ uses of Ask.SMILE for question-generating practices". Contemporary Educational Technology, vol. 18, no. 3, 2026, ep672. https://doi.org/10.30935/cedtech/18936

Phan T, Yukhymenko-Lescroart MA, Bonk CJ. AI and critical thinking: A mixed-method exploration of students’ uses of Ask.SMILE for question-generating practices. CONT ED TECHNOLOGY. 2026;18(3):ep672. https://doi.org/10.30935/cedtech/18936