Co-designing AI with youth partners: Enabling ideal classroom relationships through a novel AI relational privacy ethical framework

In recent years, the design of AI-based tools for educational spaces have been largely driven by researchers who impart their past expertises, experiences, and perspectives in the design process. While this typically leads to technically feasible designs and are often well-grounded in theories of learning, youth agency is typically limited in this process. In this paper, we argue that designers have a significant ethical responsibility to incorporate youth voices – in particular, their dreams and concerns – into the design of AI tools starting from conception. This need is particularly important as new applications for AI, such as AI-supported collaboration, introduce new surveillance vectors into classroom spaces. Drawing from recent scholarship which advances ethics and relationality in participatory co-design with youth, we introduce a co-design methodology in which youth are supported in imagining expansive technical possibilities for K-12 public schools, grounded within affordances, limitations, and tradeoffs of AI and machine learning techniques. This approach is demonstrated through our Learning Futures Workshop, which brought together 30 historically minoritized youth in conversation with experts in both education and technology. Through detailed case study on the enactment of the workshop, including a thematic analysis of the activities the youth engaged in and their outputs, we identified new, expansive relational possibilities for AI, ethical commitments to support the design, and finally, developed a novel AI Relational Privacy ethical framework that supports the design of new collaborative AI platforms. We conclude by connecting these findings and frameworks to the design of newly enacted AI-based applications and underlying data infrastructures.