GenAI Role-Play for Professional Education: Investigating and Evaluating Generative AI-Enabled Role-Play for Competency-Based Education and Professional Readiness

Evaluating whether AI-powered role-play simulations can develop the professional competencies that conventional teaching struggles to scale.

About this project

This project investigates the efficacy of generative AI-powered role-play simulations as a scalable method for developing professional competencies in Law and Forensic Science. Building on two prototype simulations already deployed via the ScholAIstic platform (a cross-examination chatbot for courtroom skills and a patient simulation chatbot for empathic communication), the study will rigorously evaluate whether AI-mediated practice can produce measurable gains that traditional peer role-play cannot match at scale.

The research adopts a mixed-methods design combining quasi-experimental and experimental conditions across approximately 500 students over three semesters. Competency development is assessed through validated, domain-specific rubrics, self-efficacy inventories, and platform-generated engagement metrics, triangulated with qualitative interview data.

A key methodological contribution is the development and validation of assessment instruments specifically adapted for chat-based AI simulations, filling a gap in current tools designed for conventional settings.

The project is grounded in a four-phase framework encompassing scenario identification, prompt engineering, conversational practice, and rubric-guided feedback, with scaffolding informed by Vygotsky’s zone of proximal development and Kolb’s experiential learning theory.

TRF project overview