Exploring the design of LLM-powered question generation for deaf and hard of hearing learners

Deaf and Hard of Hearing (DHH) learners face unique learning challenges, often due to a lack of customized educational materials that address their specific needs. This study explores the potential of Large Language Models (LLMs) to generate personalized quiz questions to enhance the video-based learning experiences of DHH students. We iteratively designed, developed, and evaluated a prototype that combines LLMs to generate questions with the characteristics of the DHH learners in focus. The two unique question generation strategies include Visual Questions, which identify video segments where visual information might be misinterpreted, and Emotion Questions, which highlight moments where previous DHH learners experienced video learning difficulties manifested in emotional responses. Through user studies with undergraduate DHH students, we evaluated the effectiveness of these LLM-powered questions generated to support the learning experience. Our findings indicate that while LLMs offer significant potential for personalized learning, challenges remain in the human-machine interaction caused by inherent accessibility issues between text-based AI prompts and DHH students’ sign-based first/native languages. The study highlights the importance of considering language diversity and culture in the design of LLM-based educational technology.