AI-based teaching evaluations: How well do they reflect student perceptions?

This study presents an innovative solution for evaluating university-level teaching quality using artificial intelligence (AI), focusing on key aspects such as clarity of explanation and lecture structure. Traditional student surveys, while valuable, are often subject to biases and lack the necessary granularity, creating a need for objective, scalable solutions that provide consistent results. We propose an automated framework utilizing advanced natural language processing (NLP) models to assess teaching quality based on lecture transcripts. The methodology combines AI-driven transcription, machine learning-based assessments, and correlation with institutional student evaluations to deliver reliable and reproducible measures of teaching effectiveness. The study analyzes 32 courses from 2017 to 2023, covering 1,222 hours of lecture video, and finds that AI assessments align significantly with student evaluations, particularly in terms of lecture structure and logical flow, though the alignment is weaker for clarity of explanation. These findings underscore the reliability of AI evaluations and suggest that they can serve as a complementary tool to traditional student feedback, offering objective, scalable insights into teaching quality. The study also highlights the limitations, such as reliance on transcribed text and the exclusion of non-verbal elements, indicating the need for multimodal AI models in future research. Finally, the paper suggests groundbreaking ideas for integrating AI into educational systems, with the potential to enhance teaching evaluation processes, making them more objective, accessible, and cost-effective, ultimately transforming the way teaching quality is assessed in academic institutions.