Co-PI: Dr. Steven Pan (NUS Psychology), Dr. Liu Liu (AICET)
About This Project
How do students in professional fields like law and forensic science gain realistic practice in high-stakes encounters such as client interviews, expert testimony, and cross-examination, when such opportunities are scarce in traditional classrooms? Peer role-play offers some answer, but it doesn’t scale, and its outcomes are notoriously hard to measure.
This project asks whether generative AI can change that. Building on two prototype simulations already deployed via the ScholAIstic platform, we are running a two-year, mixed-methods study with approximately 500 students across quasi-experimental and experimental conditions. Competency development is assessed through validated domain-specific rubrics, self-efficacy inventories, platform-generated engagement metrics, and qualitative interviews, so that conclusions rest on triangulated evidence rather than any single signal.
Our aim is twofold: to test whether AI-mediated practice produces measurable competency gains that peer role-play cannot match at scale, and to develop the assessment instruments and design principles that any rigorous evaluation of AI role-play in professional education will need.
Research Questions
Methods
Mixed-Methods Design
Quasi-Experimental
Rubric-Based Assessment
Qualitative Interviews
Key Contributions
A key methodological contribution is the development and validation of assessment instruments specifically adapted for AI-powered role-play simulations in professional education, filling a gap in current tools designed for conventional settings. The project is grounded in a four-phase framework encompassing scenario identification, prompt engineering, conversational practice, and rubric-guided feedback, with scaffolding informed by Vygotsky’s zone of proximal development and Kolb’s experiential learning theory.
Publications
[1] Liu, L., Mun, N. S., Low, K., & Leong, B. (2025). Simulating Professional Workplaces: A Pedagogical Framework for Generative AI-Powered Role-Play for Competency-Based Education. In AI4X 2025 International Conference.
[2] Liu, L., Ng, S. M., Herwana, A. S., & Yeo, M. (2025). The Educator as an Embedded AI Designer: Faculty Agency in Generative AI-Enhanced Pedagogy. In 2025 International Conference on Educational Technology Management (ICETM) (pp. 298-303). IEEE.