The effect of an agent tutor’s integration of cognitive and emotional gestures on cognitive load, motivation, and achievement

Soonri Choi; Soomin Kang; Kyungmin Lee; Hongjoo Ju; Jihoon Song

doi:10.30935/cedtech/14101

Soonri Choi ¹ ² ^* , Soomin Kang ², Kyungmin Lee ¹ ², Hongjoo Ju ¹, Jihoon Song ¹ ^*

More Detail

CONT ED TECHNOLOGY, Volume 16, Issue 1, Article No: ep491

https://doi.org/10.30935/cedtech/14101

Submitted: 20 August 2023, Published Online: 05 January 2024

OPEN ACCESS 613 Views 545 Downloads

Download Full Text (PDF)

Abstract

This study proposes that the gestures of an agent tutor in a multimedia learning environment can generate positive and negative emotions in learners and influence their cognitive processes. To achieve this, we developed and integrated positive and negative agent tutor gestures in a multimedia learning environment directed by cognitive gestures. The effects of emotion type on cognition were examined in terms of cognitive load, learning motivation, and achievement. The subjects were 46 university students in Gyeonggi Province, South Korea. The students were divided into three learner groups: cognition, cognition + negative emotion, and cognition + positive emotion. The learners watched a tutorial lecture on the Notion note-taking app by an agent tutor. Data analysis was conducted using one-way ANOVA to determine the cognitive load, learning motivation, and achievement. The results showed that the positive emotion design was more effective in terms of intrinsic cognitive load, learning motivation, and achievement but had a higher extrinsic cognitive load. However, even the negative + passive group showed more positive learning than the cognition group. Although this study focused on gestures by an agent tutor, it implies that such gestures in multimedia learning contexts must be informed by emotional as well as cognitive design to provide a more meaningful learning experience.

Keywords: agent tutor, gesture effect, cognitive theory model, cognitive-affective model

References

Atkinson, R. K. (2002). Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology, 94(2), 416-427. https://doi.org/10.1037/0022-0663.94.2.416
Ba, S., Stein, D., Liu, Q., Long, T., Xie, K., & Wu, L. (2021). Examining the effects of a pedagogical agent with dual-channel emotional cues on learner emotions, cognitive load, and knowledge transfer performance. Journal of Educational Computing Research, 59(6), 1114-1134. https://doi.org/10.1177/0735633121992421
Baylor, A. L., & Kim, S. (2009). Designing nonverbal communication for pedagogical agents: When less is more. Computers in Human Behavior, 25(2), 450-457. https://doi.org/10.1016/j.chb.2008.10.008
Baylor, A. L., & Kim, Y. (2005). Simulating instructional roles through pedagogical agents. International Journal of Artificial Intelligence in Education, 15(2), 95-115. https://doi.org/10.1007/s40593-015-0055-y
Bless, H., Clore, G., Schwarz, N., Golisano, V., Rabe, C., & Wolk, M. (1996). Mood and the use of scripts: Does a happy mood really lead to mindlessness? Journal of Personality and Social Psychology, 71(4), 665-679. https://doi.org/10.1037/0022-3514.71.4.665
Chen, O., Castro-Alonso, J. C., Paas, F., & Sweller, J. (2018). Extending cognitive load theory to incorporate working memory resource depletion: Evidence from the spacing effect. Educational Psychology Review, 30(2), 483-501. https://doi.org/10.1007/s10648-017-9426-2
Chen, Z. H. (2012). We care about you: Incorporating pet characteristics with educational agents through reciprocal caring approach. Computers & Education, 59(4), 1081-1088. https://doi.org/10.1016/j.compedu.2012.05.015
Cowan, N. (2014). Working memory underpins cognitive development, learning, and education. Educational Psychology Review, 26(2), 197-223. https://doi.org/10.1007/s10648-013-9246-y
Dinçer, S., & Doğanay, A. (2017). The effects of multiple-pedagogical agents on learners’ academic success, motivation, and cognitive load. Computers & Education, 111, 74-100. https://doi.org/10.1016/j.compedu.2017.04.005
Fredrickson, B. L. (2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American Psychologist, 56(3), 218-226. https://doi.org/10.1037/0003-066X.56.3.218
Fredrickson, B. L. (2003). The value of positive emotions: The emerging science of positive psychology is coming to understand why it’s good to feel good. American Scientist, 91(4), 330-335. https://doi.org/10.1511/2003.26.330
Heidig, S., & Clarebout, G. (2011). Do pedagogical agents make a difference to student motivation and learning? Educational Research Review, 6(1), 27-54. https://doi.org/10.1016/j.edurev.2010.07.004
Heller, B., & Procter, M. (2010). Animated pedagogical agents and immersive worlds: Two worlds colliding. In G. Veletsianos (Ed.), Emerging technologies in distance education (pp. 301-316). Edmonton.
Horovitz, T., & Mayer, R. E. (2021). Learning with human and virtual instructors who display happy or bored emotions in video lectures. Computers in Human Behavior, 119, 106724. https://doi.org/10.1016/j.chb.2021.106724
Kim, Y., & Baylor, A. L. (2016). based design of pedagogical agent roles: A review, progress, & recommendations. International Journal of Artificial Intelligence in Education, 26(1), 160-169. https://doi.org/10.1007/s40593-015-0055-y
Lawson, A. P., & Mayer, R. E. (2022). Does the emotional stance of human and virtual instructors in instructional videos affect learning processes and outcomes? Contemporary Educational Psychology, 70, 102080. https://doi.org/10.1016/j.cedpsych.2022.102080
Lawson, A. P., Mayer, R. E., Adamo-Villani, N., Benes, B., Lei, X., & Cheng, J. (2021). Do learners recognize and relate to the emotions displayed by virtual instructors? International Journal of Artificial Intelligence in Education, 31(1), 134-153. https://doi.org/10.1007/s40593-021-00238-2
LeDoux, J. E., & Brown, R. (2017). A higher-order theory of emotional consciousness. Proceedings of the National Academy of Sciences, 114(10), E2016-E2025. https://doi.org/10.1073/pnas.1619316114
Leppink, J., Paas, F., Van der Vleuten, C. P., Van Gog, T., & Van Merriënboer, J. J. (2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods, 45(4), 1058-1072. https://doi.org/10.3758/s13428-013-0334-1
Li, W., Wang, F., Mayer, R. E., & Liu, H. (2019). Getting the point: Which kinds of gestures by pedagogical agents improve multimedia learning? Journal of Educational Psychology, 111(8), 1382-1395. https://doi.org/10.1037/edu0000352
Li, W., Wang, F., Mayer, R. E., & Liu, T. (2022). Animated pedagogical agents enhance learning outcomes and brain activity during learning. Journal of Computer Assisted Learning, 38(3), 621-637. https://doi.org/10.1111/jcal.12634
Lin, L., Atkinson, R. K., Christopherson, R. M., Joseph, S. S., & Harrison, C. J. (2013). Animated agents and learning: Does the type of verbal feedback they provide matter? Computers & Education, 67, 239-249. https://doi.org/10.1016/j.compedu.2013.04.017
Lozano, S. C., & Tversky, B. (2006). Communicative gestures facilitate problem solving for both communicators and recipients. Journal of Memory and Language, 55(1), 47-63. https://doi.org/10.1016/j.jml.2005.09.002
Martha, A. S. D., & Santoso, H. B. (2019). The design and impact of the pedagogical agent: A systematic literature review. Journal of Educators Online, 16(1). https://doi.org/10.9743/jeo.2019.16.1.8
Mayer, R. E. (2020a). Multimedia learning. Cambridge University Press.
Mayer, R. E. (2020b). Searching for the role of emotions in e-learning. Learning and Instruction, 70, 101213. https://doi.org/10.1016/j.learninstruc.2019.05.010
Mayer, R. E. (2021). Evidence-based principles for how to design effective instructional videos. Journal of Applied Research in Memory and Cognition, 10(2), 229-240. https://doi.org/10.1016/j.jarmac.2021.03.007
Moreno, R., Reislein, M., & Ozogul, G. (2010). Using virtual peers to guide visual attention during learning: A test of the persona hypothesis. Journal of Media Psychology, 22(2), 52-60. https://doi.org/10.1027/1864-1105/a000008
Paas, F., & van Merriënboer, J. J. G. (2020). Cognitive-load theory: Methods to manage working memory load in the learning of complex tasks. Current Directions in Psychological Science, 29(4), 394-398. https://doi.org/10.1177/0963721420922183
Park, S. (2016). Virtual avatar as an emotional scaffolding strategy to promote interest in online learning environment. In S. Y. Tettegah, & M. Gartmeier (Eds.), Emotions, technology, design, and learning (pp. 201-224). Academic Press. https://doi.org/10.1016/B978-0-12-801856-9.00010-4
Pessoa, L. (2008). On the relationship between emotion and cognition. Nature Reviews Neuroscience, 9(2), 148-158. https://doi.org/10.1038/nrn2317
Petri, H. L., & Govern, J. M. (2012). Motivation: Theory, research, and application. Cengage Learning.
Phelps, E. A. (2004). Human emotion and memory: Interactions of the amygdala and hippocampal complex. Current Opinion in Neurobiology, 14(2), 198-202. https://doi.org/10.1016/j.conb.2004.03.015
Pi, Z., Liu, R., Ling, H., Zhang, X., Wang, S., & Li, X. (2022). The emotional design of an instructor: Body gestures do not boost the effects of facial expressions in video lectures. Interactive Learning Environments. https://doi.org/10.1080/10494820.2022.2105898
Plass, J. L., & Kalyuga, S. (2019). Four ways of considering emotion in cognitive load theory. Educational Psychology Review, 31(2), 339-359. https://doi.org/10.1007/s10648-019-09473-5
Plass, J. L., & Kaplan, U. (2016). Emotional design in digital media for learning. In S. Tettegah, & M. Gartmeier (Eds.), Emotions, technology, design, and learning (pp. 131-162). Academic Press. https://doi.org/10.1016/B978-0-12-801856-9.00007-4
Plass, J. L., Homer, B. D., MacNamara, A., Ober, T., Rose, M. C., Pawar, S., Hovey, C, M., & Olsen, A. (2020). Emotional design for digital games for learning: The effect of expression, color, shape, and dimensionality on the affective quality of game characters. Learning and Instruction, 70, 101194. https://doi.org/10.1016/j.learninstruc.2019.01.005
Reeves, B., & Nass, C. (1996). The media equation. Cambridge University Press.
Reyes, M. R., Brackett, M. A., Rivers, S. E., White, M., & Salovey, P. (2012). Classroom emotional climate, student engagement, and academic achievement. Journal of Educational Psychology, 104(3), 700-712. https://doi.org/10.1037/a0027268
Rolland, R. G. (2012). Synthesizing the evidence on classroom goal structures in middle and secondary schools: A meta-analysis and narrative review. Review of Educational Research, 84(4), 396-435. https://doi.org/10.3102/0034654312464909
Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110(1), 145-172. https://doi.org/10.1037/0033-295X.110.1.145
Ryan, R. M., Connell, J. P., & Plant, R. W. (1990). Emotions in nondirected text learning. Learning and Individual Differences, 2(1), 1-17. https://doi.org/10.1016/1041-6080(90)90014-8
Schunk, D. H. (2011). Learning theories: An educational perspective. Pearson.
Sharot, T., & Phelps, E. A. (2004). How arousal modulates memory: Disentangling the effects of attention and retention. Cognitive, Affective, & Behavioral Neuroscience, 4(3), 294-306. https://doi.org/10.3758/CABN.4.3.294
Sweller, J. (2020). Cognitive load theory and educational technology. Educational Technology Research and Development, 68(1). https://doi.org/10.4135/9781483346397.n55
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. Springer. https://doi.org/10.1007/978-1-4419-8126-4
Sweller, J., van Merriënboer, J. J. G., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31(2), 261-292. https://doi.org/10.1007/s10648-019-09465-5
Tsai, J., Bowring, E., Marsella, S., Wood, W., & Tambe, M. (2012). A study of emotional contagion with virtual characters [Paper presentation]. The International Conference on Intelligent Virtual Agents. https://doi.org/10.1007/978-3-642-33197-8_8
Um, E., Plass, J. L., Hayward, E. O., & Homer, B. D. (2012). Emotional design in multimedia learning. Journal of Educational Psychology, 104(2), 485-498. https://doi.org/10.1037/a0026609
van der Meij, H., van der Meij, J., & Harmsen, R. (2015). Animated pedagogical agents effects on enhancing student motivation and learning in a science inquiry learning environment. Educational Technology Research and Development, 63(3), 381-403. https://doi.org/10.1007/s11423-015-9378-5
Veletsianos, G. (2012). How do learners respond to pedagogical agents that deliver social-oriented non-task messages? Impact on student learning, perceptions, and experiences. Computers in Human Behavior, 28(1), 275-283. https://doi.org/10.1016/j.chb.2011.09.010
Veletsianos, G., & Russell, G. S. (2014). Pedagogical agents. In J. M. Spector, M. D. Merrill, Jan Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 759-769). Springer. https://doi.org/10.1007/978-1-4614-3185-5_61
Wang, F. X., Li, W. J., Mayer, R. E., & Liu, H. S. (2018). Animated pedagogical agents as aids in multimedia learning: Effects on eye-fixations during learning and learning outcomes. Journal of Educational Psychology, 110, 250-268. https://doi.org/10.1037/edu0000221
Xie, H., Mayer, R. E., Wang, F., & Zhou, Z. (2019). Coordinating visual and auditory cueing in multimedia learning. Journal of Educational Psychology, 111(2), 235-255. https://doi.org/10.1037/edu0000285
Yung, H. I., & Paas, F. (2015). Effects of cueing by a pedagogical agent in an instructional animation: A cognitive load approach. Journal of Educational Technology & Society, 18(3), 153-160. https://doi.org/10.1080/10494820.2020.1857784

Citation

Choi, S., Kang, S., Lee, K., Ju, H., & Song, J. (2024). The effect of an agent tutor’s integration of cognitive and emotional gestures on cognitive load, motivation, and achievement. Contemporary Educational Technology, 16(1), ep491. https://doi.org/10.30935/cedtech/14101

Choi, S., Kang, S., Lee, K., Ju, H., and Song, J. (2024). The effect of an agent tutor’s integration of cognitive and emotional gestures on cognitive load, motivation, and achievement. Contemporary Educational Technology, 16(1), ep491. https://doi.org/10.30935/cedtech/14101

Choi S, Kang S, Lee K, Ju H, Song J. The effect of an agent tutor’s integration of cognitive and emotional gestures on cognitive load, motivation, and achievement. CONT ED TECHNOLOGY. 2024;16(1), ep491. https://doi.org/10.30935/cedtech/14101

Choi, Soonri, Soomin Kang, Kyungmin Lee, Hongjoo Ju, and Jihoon Song. "The effect of an agent tutor’s integration of cognitive and emotional gestures on cognitive load, motivation, and achievement". Contemporary Educational Technology 2024 16 no. 1 (2024): ep491. https://doi.org/10.30935/cedtech/14101

Choi, Soonri et al. "The effect of an agent tutor’s integration of cognitive and emotional gestures on cognitive load, motivation, and achievement". Contemporary Educational Technology, vol. 16, no. 1, 2024, ep491. https://doi.org/10.30935/cedtech/14101

Choi S, Kang S, Lee K, Ju H, Song J. The effect of an agent tutor’s integration of cognitive and emotional gestures on cognitive load, motivation, and achievement. CONT ED TECHNOLOGY. 2024;16(1):ep491. https://doi.org/10.30935/cedtech/14101