Investigating and Evaluating Generative AI-Enabled Role-Play for Competency-Based Education and Professional Readiness
About This Project
This project investigates the efficacy of generative AI-powered role-play simulations as a scalable method for developing professional competencies in Law and Forensic Science.
Building on two prototype simulations already deployed via the ScholAIstic platform (a cross-examination chatbot for courtroom skills and a patient simulation chatbot for empathic communication), the study will rigorously evaluate whether AI-mediated practice can produce measurable gains that traditional peer role-play cannot match at scale.
The research adopts a mixed-methods design combining quasi-experimental and experimental conditions across approximately 500 students over three semesters. Competency development is assessed through validated, domain-specific rubrics, self-efficacy inventories, and platform-generated engagement metrics, triangulated with qualitative interview data.
A key methodological contribution is the development and validation of assessment instruments specifically adapted for chat-based AI simulations, filling a gap in current tools designed for conventional settings. The project is grounded in a four-phase framework encompassing scenario identification, prompt engineering, conversational practice, and rubric-guided feedback, with scaffolding informed by Vygotsky’s zone of proximal development and Kolb’s experiential learning theory.
Research Questions
Publications
Simulating Professional Workplaces: A Pedagogical Framework for Generative AI-Powered Role-Play for Competency-Based Education
This paper presents a conceptual framework that uses generative AI to implement role-play in the classroom and bridge the gap between academic instruction and workplace readiness. Focusing on domain-specific problem identification, scenario-focused prompt engineering, interactive practice, and automated rubric-guided feedback, the framework provides a guided approach for integrating immersive simulations into professional education across disciplines including law and nursing.
The Educator as an Embedded AI Designer: Faculty Agency in Generative AI-Enhanced Pedagogy
This paper introduces the Embedded AI Designer Framework, where educators serve as co-designers who incorporate disciplinary reasoning, ethical norms, and pedagogical intent into GenAI systems. The framework is validated across three disciplines at NUS — legal education, nursing education, and computer science — demonstrating how faculty agency is exercised through three stages: Intentional Design, Orchestrated Mediation, and Reflective Alignment.
Co-I:
Co-I:
Lead PI:
Steven Pan (NUS Psychology)
Liu Liu
Ben Leong Wing Lup