An AI-enhanced pattern recognition approach to temporal and spatial analysis of children's embodied interactions
Abstract
Multimodal video analysis is a complex and time-consuming process for a researcher; it entails capturing, watching, and re-watching video data to identify which segments best inform or address the questions that drive the research. Modern AI applications can alleviate the challenges that arise during the fine-grained analysis of learners' multimodal interactions captured through video. In this study, we present a supervised approach to training a deep neural network to analyze children's computational thinking (CT) captured through multimodal video data. The approach first uses a set of images extracted from video data to train the AI to map them to labels generated using a priori theory. Confusion matrices were used to establish the performance of the AI by comparing AI predictions to human analysis on a validation set of data. The findings suggested that the AI classified several aspects of children's CT in a way that was highly consistent with human analysis, demonstrating how the AI could serve as an additional team member during multimodal analysis. Implications for using AI to ease the challenges of multimodal analysis of video data are discussed.
Identifier Metadata
| Identifier | 110.1142/CON.2026.01113 |
| Canonical | mdoi:110.1142/CON.2026.01113 |
| Resolver URL | https://mdoi.org/110.1142/CON.2026.01113 |
| Resource URL | Open resource |
| Document URL | Open document |
| Content Type | Article |
| Authors | Ceren Ocak, Theodore J. Kopcha, Raunak Dey |
| Year | 2024 |
| Depositor | Convergence Chronicles Organisation |
| Prefix | 110.1142 |
| Registered | Aug. 19, 2026 |
| Updated | Aug. 19, 2026 |
| Status | Active |
| Visibility | Public |
Cite This Identifier
APA 7th Edition
Click to copy
MLA 9th Edition
Click to copy
Chicago 17th Edition
Click to copy
BibTeX
Click to copy
Persistent Identifier
mdoi:110.1142/CON.2026.01113Click to copy