Accurate, timely, and portable: Course-agnostic early prediction of student performance from LMS logs

In higher education, providing personalized feedback and support to students is a significant challenge. Early warning systems can help by identifying both at-risk and high-performing students, allowing for timely interventions and enhanced learning opportunities. In our study, we used a year's worth of data from an information management school to build predictive models for two binary classification problems: identifying at-risk students and high-performing students. We employed traditional machine learning classifiers and long-short term memory units (LSTM), testing them at various stages of course completion. The best performance was achieved using all course data, with an AUC of 0.756 for at-risk students and 78.2% accuracy for high-performing students using Random Forest and Extremely Randomized Trees, respectively. We found that early prediction was possible as early as 25% course completion. Although LSTM showed inferior performance, it offered practical advantages for early prediction. Our findings suggest that static LMS logs can be reliable indicators of student success early in a course, and a course-agnostic time-dependent representation of the number of clicks can offer a worthwhile tradeoff between predictive performance and simplicity in implementation in some instances. These findings have important implications as they suggest the potential for automated early warning systems that can help educators identify students of interest and allocate resources where they are most needed. However, implementing these systems in real-time requires clear protocols and responsible policies. Further research should explore the generalizability of findings across different contexts and continuously evaluate their real-world effectiveness.