Context-aware prompting for collaborative problem solving skill identification

Collaborative Problem Solving (CPS) competence is widely recognized as a critical skill for the 21st century, especially in professions that emphasize communication and teamwork. CPS is a complex process involving both cognitive and social components, and numerous theoretical frameworks have been proposed for its assessment. Currently, CPS competence is typically measured by analyzing process data from simulated tasks. To facilitate a more structured analysis of the CPS process, it is necessary to code behaviors into specific CPS skills. Traditionally, this coding has been conducted manually using predefined coding schemes. However, manual coding is both time-consuming and labor-intensive, making it impractical for large-scale datasets and unsuitable for real-time applications. To overcome these limitations, researchers have developed various automated coding methods. Nonetheless, most existing methods treat each behavior as independent, overlooking the natural dependencies among behaviors in the CPS process. To address this shortcoming, we propose a novel framework called Context-Aware Prompting for CPS (CAP4CPS), which enables automated CPS skill coding by explicitly modeling contextual dependencies. Our approach first employs a context extraction module to retrieve dialogue histories that reflect the cognitive and social aspects of CPS. Next, a context-infused prompting module generates context-rich prompts to guide a pre-trained language model in producing representations of cognitive and social abilities. Finally, an ability fusion module integrates these representations into a unified CPS competency representation, which is then used to predict the corresponding CPS skills. Extensive experiments on two CPS task datasets demonstrate the effectiveness and robustness of the CAP4CPS.