NUS Trial

LLM Adoption, Attitudes and Behaviour at Scale

About This Project

The NUS Trial is a longitudinal, cross-faculty study that investigates how university students adopt, adapt to, and are shaped by Large Language Models over the course of their studies. Conducted across the NUS School of Computing, Faculty of Arts and Social Sciences (FASS), and NUS College, the study is distinctive in combining two streams of evidence: periodic surveys that track students’ self-reported usage patterns, attitudes, perceived risks, and epistemic habits over time, and direct analysis of conversation logs captured through ScholAIstic, AICET’s purpose-built platform. This dual-stream design allows the project to go beyond what students say they do and examine what they actually do, surfacing the gap between reported and real behaviour that survey-only studies routinely miss.

ScholAlstic Platform

By analysing the content, structure, and trajectory of student-AI conversations at scale, the project asks deeper questions: How do students frame their requests to AI? Do they engage critically or deferentially? And do their patterns of engagement shift as they gain more experience with these tools? Spanning multiple disciplines, the NUS Trial is one of the few studies positioned to track how LLM adoption unfolds not as a snapshot, but as a process.

Research Questions

How do LLM usage patterns and attitudes differ across academic disciplines, and what factors drive cross-faculty variation?

To what extent do students’ self-reported LLM behaviours align with their actual usage as captured in conversation logs, and what does this gap reveal?

How do students’ patterns of AI engagement evolve over time as they gain more experience with these tools?

Team Members:

Lead PI:

Research Period:

Gladia Hotan, Liu Liu, Marcus Yeo, Lucius Khor, Olivia

Ben Leong Wing Lup

2024 – Present