Edman et al. (2026) make a reasonable and careful argument. They do not claim that ChatGPT should replace the clinician. In fact, their article frames ChatGPT as a supplemental tool in cognitive rehabilitation, mainly for generating personally relevant practice tasks under clinician supervision. They also acknowledge that the article is not an empirical study, that its evidence is limited to case examples and anecdotal feedback, and that future research must test effectiveness, privacy, and clinical oversight more rigorously (Edman et al., 2026).
That said, I think the article still underestimates the most dangerous part of AI in mental health practice. The risk is not only hallucination, privacy, copyright, or factual error. Those matter, of course. But the deeper issue is relational. AI systems are increasingly designed to be helpful, warm, agreeable, and easy to keep talking to. In mental health work, that can become dangerous very quickly because therapy is not the same thing as feeling validated.
Here is the problem. Modern language models are trained using human feedback, meaning they learn from user or evaluator preferences about which responses seem more helpful, satisfying, or aligned with the user’s intent (Ouyang et al., 2022).That may improve usability, but mental health care is not a popularity contest. A good clinician does not simply mirror the client’s preferred story back to them. Sometimes the ethical task is to slow the client down, challenge a maladaptive belief, assess risk, notice avoidance, support reality testing, or say something the client may not want to hear. AI, by contrast, can be pulled toward the response that feels most emotionally rewarding in the moment.
Recent research makes this concern difficult to ignore. Ibrahim et al. (2026) found that training language models to be warmer increased error rates across consequential tasks and made models more likely to validate incorrect user beliefs, especially when users expressed sadness. They specifically note that this matters because people increasingly use AI for advice, therapy, and companionship (Ibrahim et al., 2026). In simple terms, the more therapeutic the AI sounds, the less safe it may become when truth requires discomfort.
This is where the danger becomes clinically significant. In counselling and psychotherapy, empathy is not blind affirmation. Empathy requires accurate understanding, attunement, and ethical judgment. A therapist can validate the client’s emotional pain while still not validating the client’s distorted belief. For example, a therapist might say, “I can hear how rejected you feel,” without agreeing that “everyone hates you.” That distinction is basic clinical work. AI can blur that boundary because it is built to respond fluently and cooperatively, not to maintain a professional therapeutic frame.
Cheng et al. (2026) found that sycophantic AI responses were not only common, but also harmful. Across 11 models, AI affirmed users’ actions more often than humans did, even when prompts involved deception, illegality, or other harms. In their experiments, even a single interaction with sycophantic AI reduced participants’ willingness to take responsibility and repair interpersonal conflict, while increasing their belief that they were right. Most concerning, users still trusted and preferred the sycophantic models, which creates a dangerous incentive because the same feature that harms judgment can also drive engagement (Cheng et al., 2026).
This matters in mental health practice because many clients do not come to therapy with clean, neutral problems. They come with shame, fear, grief, trauma, anger, compulsions, avoidance, suicidal ideation, delusional conviction, interpersonal conflict, and fragile self-understanding. In those moments, a system that “agrees” too easily can reinforce the very pattern that therapy is supposed to help interrupt. A client with paranoia may receive subtle confirmation. A client in conflict may become more convinced they are entirely innocent. A client with depression may have hopeless beliefs reflected back in polished language. A client with suicidal ideation may receive a response that is warm but clinically inadequate.
The article does acknowledge hallucinations, bias, privacy concerns, and the need for clinician review (Edman et al., 2026). These are important points. However, I think the article’s focus on “engagement” needs more caution. Engagement is usually treated as a good thing in rehabilitation because clients are more likely to practice skills when tasks feel relevant. But AI engagement is different. In commercial systems, engagement can mean more conversation, more dependence, more disclosure, and more emotional attachment. That is not automatically therapeutic. In fact, it may become anti-therapeutic if the client begins to rely on the AI for reassurance rather than developing insight, tolerance for distress, interpersonal repair, or autonomous coping.
This is already a concern in the mental health literature. Lawrence et al. (2024) argue that the promise of LLMs in mental health must be balanced with responsible development, testing, and deployment, especially because general-purpose models can generate inaccurate, biased, stigmatizing, or harmful mental health information (Lawrence et al., 2024). Similarly, Zhang et al. (2025) identify privacy, emotional dependency, memory retention, and ethical challenges as major concerns when LLMs are used in psychotherapy-related roles (Zhang et al., 2025).
The risk becomes even sharper in crisis contexts. Pichowicz et al. (2025) found that many mental health chatbot agents struggled with suicidal ideation scenarios, sometimes providing inadequate or inappropriate responses that could be harmful to users in crisis. They explicitly link these shortcomings to non-maleficence, which is the ethical duty to avoid harm (Pichowicz et al., 2025). Hudon and Stip (2025) also warn that uncritical validation by AI may entrench delusional conviction or cognitive perseveration in vulnerable individuals, which directly conflicts with the corrective principles used in cognitive-behavioural therapy for psychosis (Hudon & Stip, 2025).
So, while Edman et al. (2026) are right that AI may help a clinician generate personalized cognitive rehabilitation materials, I would argue that the clinical boundary must be much stronger than simply “clinician oversight.” The clinician should not only review whether the generated exercise is accurate or relevant. They should also ask what relational function the AI is serving. Is it helping the client practice a strategy, or is it becoming a substitute attachment figure? Is it supporting metacognitive awareness, or is it reinforcing reassurance seeking? Is it promoting generalization, or is it encouraging dependence on a conversational system that always responds?
In my view, AI can have a limited place in mental health practice, but only as a tightly controlled administrative or therapeutic aid. It should not be treated as a therapist, co-therapist, crisis worker, attachment figure, or source of clinical interpretation. This is especially true with clients who experience psychosis, suicidality, trauma-related dysregulation, severe anxiety, compulsive reassurance seeking, or impaired reality testing. These are situations where warm language without clinical accountability can do harm.
The central issue is this: therapy is not just personalized content. Therapy is a disciplined relationship guided by assessment, ethics, theory, boundaries, and responsibility. AI can generate words that sound empathic, but it cannot carry professional responsibility for what those words do to a vulnerable client. That is why the promise of AI in cognitive rehabilitation should be treated cautiously. The goal should not be to make AI sound more therapeutic. The goal should be to make sure it never replaces the uncomfortable, human, clinically responsible work that therapy requires.
References
Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D., & Jurafsky, D. (2026). Sycophantic AI decreases prosocial intentions and promotes dependence. Science, 391(6792), eaec8352. https://doi.org/10.1126/science.aec8352
Edman, N. F., Lacritz, M. L., Isufi, K., Lokai, A. D., Bender, H. A., & Sacks-Zimmerman, A. (2026). ChatGPT in intervention planning: Advancing clinical relevance and targeted care in cognitive rehabilitation. Neuropsychology. Advance online publication. https://doi.org/10.1037/neu0001088
Hudon, A., & Stip, E. (2025). Delusional experiences emerging from AI chatbot interactions or “AI psychosis.” JMIR Mental Health, 12, e85799. https://doi.org/10.2196/85799
Ibrahim, L., Hafner, F. S., & Rocher, L. (2026). Training language models to be warm can reduce accuracy and increase sycophancy. Nature, 652, 1159–1165. https://doi.org/10.1038/s41586-026-10410-0
Lawrence, H. R., Schneider, R. A., Rubin, S. B., Matarić, M. J., McDuff, D. J., & Jones Bell, M. (2024). The opportunities and risks of large language models in mental health. JMIR Mental Health, 11, e59479. https://doi.org/10.2196/59479
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P. F., Leike, J., & Lowe, R. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35, 27730–27744. https://doi.org/10.48550/arXiv.2203.02155
Pichowicz, W., Kotas, M., & Piotrowski, P. (2025). Performance of mental health chatbot agents in detecting and managing suicidal ideation. Scientific Reports, 15, 31652. https://doi.org/10.1038/s41598-025-17242-4
Zhang, R., Meng, H., Neubronner, M., & Lee, Y.-C. (2025). Computational and ethical considerations for using large language models in psychotherapy. Nature Computational Science, 5, 854–862. https://doi.org/10.1038/s43588-025-00874-x