Optimizing automated scoring in ILSAs with prompt compression
Abstract
Automated scoring (AS) has become increasingly prevalent in educational measurement. However, applying it to international reading assessments remains challenging, particularly due to the length and complexity of the required prompting, driven by the need to include lengthy reading passages and detailed scoring guides. Processing these lengthy inputs results in high computational costs and may impede the performance of large language models (LLMs). This study explored the potential of optimizing AS with prompt compression using OpenAI's LLM, GPT-4o. Our results show that prompt compression significantly reduces the length of reading passages and scoring guides while maintaining their essential content. Reading passages and scoring guides were compressed to approximately 18% and 15% of their original lengths, respectively. Despite this substantial compression, the AS showed remarkable performance, with an accuracy of 92.87% and a kappa score of 0.8041, closely approximating the results obtained without compression. These findings suggest optimizing AS with prompt compression can improve its efficiency and scalability, particularly in international reading assessments.
Identifier Metadata
| Identifier | 110.0901/CON.2026.00872 |
| Canonical | mdoi:110.0901/CON.2026.00872 |
| Resolver URL | https://mdoi.org/110.0901/CON.2026.00872 |
| Resource URL | Open resource |
| Document URL | Open document |
| Content Type | Article |
| Authors | Ji Yoon Jung, Ummugul Bezirhan, Matthias von Davier |
| Year | 2025 |
| Depositor | Convergence Chronicles Organisation |
| Prefix | 110.0901 |
| Registered | July 31, 2026 |
| Updated | July 31, 2026 |
| Status | Active |
| Visibility | Public |
Cite This Identifier
APA 7th Edition
Click to copy
MLA 9th Edition
Click to copy
Chicago 17th Edition
Click to copy
BibTeX
Click to copy
Persistent Identifier
mdoi:110.0901/CON.2026.00872Click to copy