Is ChatGPT a good study companion? The role of AI-generated summaries and reflective prompts in learning from educational videos

Online videos have become a central tool in modern education. Alongside this shift, Artificial Intelligence (AI) is reshaping personalized learning experiences, with generative large language models like ChatGPT offering new ways to tailor information to individual learners. Based on the Cognitive Theory of Multimedia Learning (CTML), which proposes two principles that relate to the interaction with the learning material (segmenting and generative activity), we conducted two experiments in which participants were asked to pause an educational video at times of comprehension difficulty. In Experiment 1 (N = 101), we examined whether GPT-generated summaries -introduced at self-paced pause points-result in better learning compared to video transcripts. In Experiment 2 (N = 215), we compared the role of GPT-generated summaries and GPT-generated reflective prompts. Those elicited open-ended answers from the participants. We measured retention and transfer learning, as well as mental effort, and perceived task difficulty. Contrary to our expectations, we observed no differences between AI summaries and transcripts in terms of retention and transfer outcomes. Participants showed a learning effect indicating more correct answers after watching the video, but this effect did not differ between conditions. We can especially note that the motivation to engage in the material, as well as the difficulty and length of the video, may have affected the results. As research investigating the role of AI in educational settings is still new, future research can delve into finding the optimal conditions under which AI can benefit learning outcomes.