Simulation-based psychological skills training and evaluation using large language models: A narrative review

This narrative review explores the application of large language models (LLMs) in simulation-based training for mental health practitioners. It highlights how LLMs, such as ChatGPT, can serve as simulated clients in training contexts to help increase access to training and ensure the establishment and maintaining of competencies. While LLMs are not intended to replace human-led training, they offer scalable, low-cost, and accessible opportunities for practitioners to rehearse and refine therapeutic skills in a more targeted, safe and flexible environment. They also offer the opportunity for closer analysis of therapeutic interactions including language and paralanguage dimensions of the trainee therapist.

Skills acquisition lies in enhancing trainee competency through repeated practice and targeted feedback. Skills evaluation while structured can be labour intensive and also subject to bias of the assessor, particularly around common factor competencies. LLMs offer an opportunity to provide both greater consistency and variety in client presentations, give more flexible opportunities for extensive practice and assist the student in the analysis of their performance.
Systems are being developed throughout the world that are making these opportunities a reality. LLMs like ClientBot (Tanana, 2019) have been shown to improve trainees’ use of fundamental counselling techniques such as reflections and open questions. Tools like Patient-Ψ (Wang et al, 2024) integrate cognitive models with LLMs, offering tailored feedback and diverse simulated client profiles to improve fidelity and confidence in cognitive behavioural therapy (CBT) formulation.

Studies show that incorporating expert-in-the-loop frameworks, like Roleplay-doh (Louie et al 2024), enhances realism by refining AI responses based on practitioner feedback. This co-production approach increases training relevance and safety and ensures human oversight of therapeutic training. The inclusion of counsellors-in-training in content generation may further enhance relevance and engagement.

Evaluation remains a challenge however this is being addressed by scales such as the Comprehensive Evaluation Scale for LLM-Powered Counselling Chatbots (CES-LCC) (Bolpagni and Gabrielli, 2025) with a 27-item, 9-dimension tool encompassing empathy, memory, trust, and emotional support—vital for assessing LLM realism and therapeutic utility. However, disparities persist. Current LLMs perform less effectively with adolescents, males, and individuals with lower education levels, underscoring the need for personalization that promotes equity without fostering unhealthy attachment.

In conclusion, LLMs present transformative possibilities for enhancing psychological training and assessment. They can offer realistic, scalable, and cost-effective training experiences that improve skill acquisition, confidence, and potentially clinical outcomes. Yet, their integration must be carefully managed through rigorous evaluation, ethical design, and a blended approach that preserves the interpersonal richness of traditional methods and also does not diminish the value of peer interaction during training. This paper reviews the current landscape and how it is rapidly evolving and draws on our experiences of training low-intensity CBT therapists, while considering future challenges and opportunities. Further research should prioritize standardizing frameworks, improving demographic responsiveness, and extending beyond text-based interactions to voice and video modalities. These developments could significantly enhance the preparation and ongoing development of mental health practitioners.