Session Information
02 SES 05 A, AI and Digitalisation III
Paper Session
Contribution
Communication competence is a key transversal skill for employability in vocational education and training (VET) (Calero López & Rodríguez-López, 2020) and a domain-specific competence in occupations with intensive customer contact. In the Swiss dual VET system, retail apprentices are expected to handle challenging complaint situations professionally, a competence anchored in the occupational ordinance and educational plan (SBFI, 2021; Bildung Detailhandel Schweiz, 2021). Despite this curricular emphasis, opportunities for repeated, high-quality practice remain scarce at classroom scale. Authentic role-play typically requires trained actors, dedicated rooms, and expert coaching to structure reflection. These resource requirements limit frequency and equity of access and constrain vocational schools’ capacity to offer deliberate practice with timely, task-specific feedback. Immersive roleplay with embodied conversational agents offers a plausible response to this gap. Virtual role-playing can provide repeated, situated practice with standardised starting conditions and comparable difficulty levels, while VR can increase experiential fidelity through embodiment, social presence, and contextual cues (Othlinghaus-Wulhorst et al., 2019). Recent advances in generative AI extend this potential by enabling unscripted, adaptive dialogue without predefined branching scripts and by supporting structured post-simulation coaching dialogues grounded in the preceding interaction (Okonkwo & Ade-Ibijola, 2021; Li et al., 2025). Conversational agents are also increasingly used to scaffold post-simulation debriefing, where structured reflection and transfer are consolidated. Recent evidence from a VR counselling simulation suggests that chatbot-guided debriefing can yield learning gains comparable to human-led debriefing (Evangelou et al., 2026).
This paper presents an AI-mediated communication training and coaching system for retail apprentices, implemented across three modalities (audio-only, desktop, XR). The training unfolds in three pedagogically sequenced phases supported by two distinct AI personas. The coach persona, Gisela Dacosta, is an AI coach and expert in retail sales communication who conducts the introduction and reflection phases. The customer persona, Klaus Trubel, is an adversarial AI customer (a middle-aged man with biases against Generation Z) who drives the complaint-handling simulation.
- In the introduction phase (about 1 minute), the coach provides a short briefing on purpose, structure, and success criteria.
- In the customer conversation phase (about 5 minutes), learners engage in an unscripted complaint dialogue with Klaus Trubel, who claims a jacket is defective and demands a substantial price reduction. A replacement is unavailable and he refuses a store voucher. His behaviour is intentionally confrontational but becomes more cooperative if the apprentice maintains a consistently professional and friendly demeanour. For classroom practicability, the interaction is time-bounded and ends after a fixed number of turns.
- In the reflection and feedback phase (about 5 minutes), the coach returns to guide structured self-evaluation: she summarises key moments, prompts sense-making and perspective taking, and provides criterion-referenced feedback aligned with the Swiss qualification procedure (e.g., offering appropriate solutions in difficult situations and creating a positive customer experience).
The prototype was developed as a design-based research (DBR) implementation study (Design-Based Research Collective, 2003; McKenney & Reeves, 2012) with two aims: (a) to lay the groundwork for sustainable classroom implementation through field-based development in close collaboration with practitioners, including a prospective educational publisher as practice partner; and (b) to derive transferable design characteristics for integrating AI-driven VR communication training into everyday VET teaching.
Method
The XR prototype (Meta Quest 3 with colour passthrough) was developed and refined through iterative DBR-oriented work that combined context analysis, co-design with practitioners and apprentices, field-based pilots, and redesign (Design-Based Research Collective, 2003; McKenney & Reeves, 2012). A context analysis with vocational practitioners informed the selection of an assessment-proximate complaint scenario and the translation of Swiss retail VET standards into explicit learning goals, success criteria, and boundary conditions for classroom use (time, setup effort, orchestration). Co-design activities with vocational teachers, retail experts, apprentices, and developers specified scenario cues, escalation dynamics, persona safeguards (role stability), and reflection scaffolds, aligning pedagogical requirements with AI and VR constraints. Successive prototypes were then piloted in vocational school settings (including formative pilots with n = 16 apprentices), informing revisions to scenario structure, customer and coach behaviour, and the reflection prompt and feedback format, with a sustained focus on practicability under authentic classroom conditions. A final summative field evaluation examined classroom-scale practicability and perceived learning value of the XR variant. In a counterbalanced within-subject study with a 3×3 Latin-square order (N = 18), apprentices completed the same task in three modalities (audio, desktop, XR); this paper reports the XR condition in detail. Quantitative baseline and post-session measures were embedded in the procedure. A baseline questionnaire (Q1, plenary) captured participant characteristics (e.g., prior VR experience and self-reported technical ability). After each modality, participants completed a post-condition questionnaire (Q2) comprising the NASA Task Load Index (NASA-TLX) and a Presence Questionnaire (0–100 ratings) to capture workload and modality-sensitive experience (e.g., naturalness and perceived control). At the end, a final questionnaire (Q3, plenary) captured overall training evaluation and adoption preferences, including modality-specific ratings, a preferred modality ranking for classroom use, and open comments. Quantitative data were analysed descriptively. Qualitative data were collected via a semi-structured group interview with all participants and a focus group (n = 6) focusing on the coaching reflection. System traces (e.g., interaction duration, interruptions, termination points) supported triangulation, including breakdowns observed in practice (e.g., reduced speech-recognition accuracy in noisier moments).
Expected Outcomes
The study provides implementation-relevant evidence on where AI-driven XR role-play adds value in VET and which design decisions support classroom adoption. Across iterations, apprentices described the unscripted customer encounter as engaging and close to vocational routines when the customer persona escalated credibly and remained role-consistent. In the summative study, post-condition ratings suggested that XR can strengthen perceived interaction naturalness and control, while workload ratings helped characterise the practical trade-offs of XR use under classroom constraints. A salient limitation for authentic lessons is robustness. In real vocational school environments, background noise and time pressure sometimes degraded speech recognition and turn-taking, which could interrupt the simulation, reduce perceived authenticity, and lead learners to discount the coach’s feedback. Learners also noted that the coaching reflection was less helpful when feedback became generic or repetitive, particularly when parts of the preceding dialogue were not captured reliably. Synthesising learner feedback, quantitative indicators (NASA-TLX and presence ratings), observations, and trace data, we derive three design principles that closely reflect the project’s original design strategies: 1. Align scenario complexity with vocational practice: Use assessment-aligned scenarios and legible task structure; prioritise persona coherence and meaningful behavioural variability over additional complexity. 2. Embed structured, contextualised AI-coach reflection immediately after practice: Anchor feedback in concrete interaction events (for example via turn-referenced cues) and provide goal-specific, criterion-linked suggestions while avoiding redundant prompts. 3. Enhance authenticity and robustness of conversational dynamics: Reduce repetition, latency, and implausible responses and provide error-tolerant fallbacks, as technical instability in XR can disproportionately undermine perceived usefulness and scalability in classrooms.
References
Bildung Detailhandel Schweiz (BDS). (2021). Bildungsplan zur Verordnung über die berufliche Grundbildung für Detailhandelsfachfrau EFZ / Detailhandelsfachmann EFZ. Bern. Calero López, I., & Rodríguez-López, B. (2020). The relevance of transversal competences in vocational education and training: A bibliometric analysis. Empirical Research in Vocational Education and Training, 12, Article 12. https://doi.org/10.1186/s40461-020-00100-0 Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. https://doi.org/10.3102/0013189X032001005 Evangelou, D., Mulders, M., & Träg, K. H. (2026). Debriefing in virtual reality simulations for the development of counseling competences: Human-led or AI-guided? Technology, Knowledge and Learning. https://doi.org/10.1007/s10758-025-09941-8 Hübsch, T., Vogel-Adham, E., Vogt, A., & Wilhelm-Weidner, A. (2024). Articulating tomorrow: Large language models in the service of professional training. VDI/VDE Innovation + Technik. https://doi.org/10.25656/01:29036 Li, Z., Babar, P. P., & Peiris, R. L. (2025). Generative role-play communication training in virtual reality for autistic individuals: A study on job coach experiences in vocational training programs. In Proceedings of CHI 2025. ACM. https://doi.org/10.1145/3706598.3713507 McKenney, S., & Reeves, T. C. (2012). Conducting educational design research. Routledge. Okonkwo, C. W., & Ade-Ibijola, A. (2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2, 100033. https://doi.org/10.1016/j.caeai.2021.100033 Othlinghaus-Wulhorst, J., Mainz, A., & Hoppe, H. U. (2019). Training customer complaint management in a virtual role-playing game: A user study. In M. Scheffel et al. (Eds.), Transforming Learning with Meaningful Technologies (EC-TEL 2019), LNCS 11722 (pp. 436–449). Springer. https://doi.org/10.1007/978-3-030-29736-7_33 Peters, M., Tschöpe, T., Konheiser, S., Raecke, J., & Schnitzler, A. (2023). Development of a digital training for social and emotional competences for medical assistants in vocational education and training in Germany. Empirical Research in Vocational Education and Training, 15, Article 4. https://doi.org/10.1186/s40461-023-00143-z Staatssekretariat für Bildung, Forschung und Innovation (SBFI). (2021). Verordnung über die berufliche Grundbildung für Detailhandelsfachfrau/Detailhandelsfachmann EFZ (SR 412.101.221.40). Fedlex.
Update Modus of this Database
The current conference programme can be browsed in the conference management system (conftool) and, closer to the conference, in the conference app.
This database will be updated with the conference data after ECER.
Search the ECER Programme
- Search for keywords and phrases in "Text Search"
- Restrict in which part of the abstracts to search in "Where to search"
- Search for authors and in the respective field.
- For planning your conference attendance, please use the conference app, which will be issued some weeks before the conference and the conference agenda provided in conftool.
- If you are a session chair, best look up your chairing duties in the conference system (Conftool) or the app.