Model for Hybrid Teaching (HyTea)

AI feedback has the potential to be always available and can allow learners to practice specifically and repeatedly at their own pace. By embedding AI feedback into immersive and multimodal technologies, such as augmented and virtual reality (AR/VR) or sensor-based systems, learners can practice physical learning tasks in addition to traditional cognitive tasks. We refer to these AI systems as multimodal tutors. One scenario that must be considered when replacing human feedback is that the AI system may deviate from the learner's expectations and the instructor's learning objectives, risking becoming unethical. This challenge is known in AI ethics research as the "alignment problem" and points to the importance of developing AI systems that are consistent with human values. This project addresses the "alignment problem" in distance learning in the specific context of presentation skills training.

The goal of the HyTea project is to address this "alignment problem". The project aims to develop a multimodal tutor that trains presentation skills such as posture, pauses and intonation of the voice. The Multimodal Tutor will provide various game-like immersive scenarios on a mixed reality headset. Supported by AI-generated feedback, these scenarios will enable learners to practice independently. Human experts should be enabled to monitor the learning process asynchronously via an alignment dashboard. They should be able to evaluate the learner's performance and the quality of the AI feedback and ensure that it is correctly aligned with the learning objectives.

---

Funding: BMBF

Cooperation: Cologne Game Lab

Duration: 09/2022 - 08/2025

State: ongoing

Project Team: Nina Mouhammad, Stefan Hummel

Contact: Daniele Di Mitri