In a fictional case, a 20-year-old student comes to the consultation for insomnia. In the middle of the conversation, he says that for months he has talked every night with a chatbot, and that last week he wrote to it that he "no longer saw any point in continuing". The program replied with a generic message of support and changed the subject. He asks, a little embarrassed, whether that "counts".

It counts, and a lot. Mental health chatbots have gone from a curiosity to a daily tool for many people, and research has advanced on two fronts that barely talk to each other. On one side, randomized trials measure the effect of these programs on symptoms of depression, anxiety and stress. On the other, audit studies test how chatbots respond when someone mentions suicide or self-harm.

This text summarizes what the meta-analyses show, who took part in the studies, where the results are inconclusive and what to do in the consultation when a patient says they talk with a chatbot in a crisis.

Key points

• In a review of 13 randomized trials with young people, 11 of them pooled in the meta-analysis, chatbots reduced psychological distress with a small effect size (Hedges g of −0.28), with certainty of evidence rated as moderate by the authors (Li et al., 2025).
• In the same work, the effect on psychological well-being was inconclusive, because the confidence interval included the absence of effect (Li et al., 2025).
• In a meta-analysis of eight trials conducted in Asia, with 921 participants, chatbots reduced depressive symptoms, but the effects on anxiety and stress were inconclusive (Leung et al., 2026).
• In a test with 29 chatbots facing simulated messages of increasing suicide risk, none gave a response considered adequate by the authors' criteria (Pichowicz et al., 2025).
• Most trial participants were young people without a diagnosis, recruited in schools and universities, and almost no study followed the effects for more than a few weeks.

• What a mental health chatbot is
• What the meta-analyses in young people show
• What the studies in Asia and the broader reviews show
• Where the risk lies: the response in a crisis
• What to do when a patient says they use a chatbot
• How NEXO helps train this reasoning
• Frequently asked questions
• Take-home messages

What a mental health chatbot is

A chatbot is a program that converses with the user by text or by voice. In mental health, the review by Li and colleagues describes an evolution from rule-based systems, with predefined responses and decision trees, to artificial intelligence (AI) systems that process natural language (Li et al., 2025). The most recent ones use large language models, the same ones behind general-purpose assistants.

This distinction matters because the studies mix very different products. A rule-based chatbot follows a cognitive-behavioral therapy script written by clinicians. A generative model creates each response on the spot, and may say things nobody anticipated. And many people use, to vent, general-purpose assistants that were neither designed nor tested as a mental health intervention.

How this topic differs from symptom checkers

The post about symptom checkers deals with apps that receive a list of complaints and return hypotheses and a degree of urgency. Here the subject is different: programs that converse continuously with the person, with the intention of reducing symptoms, and what happens when that conversation reaches a crisis. The central question is no longer whether the program gets the diagnosis right but whether it helps and whether it protects.

What the meta-analyses in young people show

The systematic review with meta-analysis by Li and colleagues searched for intervention studies with chatbots for young people aged 10 to 24, published between January 2014 and September 2024 (Li et al., 2025). The authors included 29 studies, of which 13 were randomized trials; 11 of them entered the meta-analysis, and two were left out for lack of data.

The main result was a reduction in psychological distress, which combined depression, anxiety, stress and negative affect, with a Hedges g of −0.28 and a 95 percent confidence interval of −0.46 to −0.10 (Li et al., 2025). A Hedges g of around 0.2 to 0.3 is usually read as a small effect. The authors rated the certainty of this evidence as moderate, downgraded for heterogeneity between studies.

For psychological well-being, the Hedges g was 0.13, with an interval of −0.16 to 0.41 (Li et al., 2025). Because the interval includes zero, the result is inconclusive: the data are compatible with benefit, with no effect and with small harm. The certainty of this evidence was rated as low, due to risk of bias and inconsistency.

What the subgroups show

The authors ran pre-defined subgroup analyses, and each one has few studies, which calls for caution. For psychological distress, the reduction was statistically significant in clinical or subclinical populations, with g of −0.34 and an interval of −0.57 to −0.10, in five trials (Li et al., 2025). In populations without clinical symptoms, g was −0.19, with an interval of −0.48 to 0.10, in six trials, an inconclusive result.

AI-based chatbots had g of −0.36, with an interval of −0.58 to −0.14, in eight trials, while rule-based ones had g of −0.09, with an interval of −0.39 to 0.20, in four trials, an inconclusive result (Li et al., 2025). Multimodal interaction had g of −0.39 and text-only had g of −0.19 with an interval that includes zero. For well-being, the direction reverses: only rule-based chatbots showed a statistically significant effect, with g of 0.29 and an interval of 0.01 to 0.57, in three trials.

These contrasts come from comparisons between different studies, with different populations and controls, and serve to generate hypotheses, not to choose a type of chatbot.

Who was studied

Participants' ages ranged from 8 to 33 years, and men were on average 38.3 percent of the samples (Li et al., 2025). Twenty of the 29 studies recruited populations without a diagnosis, five included people with psychological symptoms, four recruited clinical populations, and most took place in schools and universities. High-income countries accounted for 62.1 percent of the studies, and the most represented were the United States, China and New Zealand, according to the authors (Li et al., 2025).

The authors themselves point out the limits: samples without a diagnosis, from developed countries, risk of bias and heterogeneity, few studies to assess long-term effects and inclusion of only publications in English and Chinese. They also note that only 14 of the 29 studies had safety measures, such as emergency assistance, crisis identification or professional follow-up.

The first trial with students

One of the most cited trials illustrates the kind of evidence behind these averages. Fitzpatrick and colleagues describe an unblinded randomized trial with 70 students aged 18 to 28 who reported symptoms of anxiety and depression, recruited through a university social network (Fitzpatrick et al., 2017). One group used a chatbot with cognitive behavioral therapy content for two weeks, and the other received an informative e-book about depression.

In the intention-to-treat analysis, the chatbot group reduced depressive symptoms measured by the PHQ-9, and the control group did not (Fitzpatrick et al., 2017). In the analysis of those who completed the study, anxiety fell in both groups, with no demonstrated advantage for the chatbot. The authors describe it as a feasibility study, with a small sample, a short intervention and no follow-up. One of the authors founded the company that created the chatbot, a conflict declared in the article.

What the studies in Asia and the broader reviews show

The review by Leung and colleagues gathered eight randomized trials, with 921 participants, conducted in China, South Korea, India, Japan and Taiwan, published between 2020 and 2023 (Leung et al., 2026). The chatbots reduced the severity of depressive symptoms by 0.46 on the authors' pooled effect measure, with a 95 percent confidence interval of −0.76 to −0.16 and high heterogeneity (Leung et al., 2026).

In the subgroups, the reduction in depression was significant in clinical populations and among Chinese participants, but not among South Korean participants, and it was significant in the comparison with bibliotherapy, but not in the comparisons with other controls (Leung et al., 2026). For anxiety, positive affect and stress, the review found no significant effect, which makes these results inconclusive.

The authors conclude that no adverse event was reported, but the information comes from only three studies that described safety (Leung et al., 2026). They list as limitations about 40 percent of the trials with high risk of bias (three of the eight), few studies with follow-up beyond four weeks and limited power in most comparisons (Leung et al., 2026).

Reviews with more trials

A larger meta-analysis gathered 48 randomized trials, with 28,071 participants from 31 countries, mixing AI-based and rule-based chatbots (Mokhtari Masoumi Alamdarloo et al., 2026). The effects, expressed as standardized mean difference, were −0.27 for depression, −0.20 for anxiety and −0.26 for stress, all with intervals that exclude zero. The authors note that most studies took place in high-income countries, that fewer than half addressed safety, human backup or monitoring of adverse events, and that long-term data are lacking.

For generative chatbots, a systematic review gathered 26 studies and a meta-analysis of 14 trials with 6314 participants (Zhang et al., 2025). The pooled effect size was 0.30 in reducing mental health problems, with a confidence interval of 0.004 to 0.59. The prediction interval, from −0.85 to 1.67, includes the absence of effect: in a new study, the result can range from harm to large benefit, which makes the prediction for a specific context inconclusive.

Source and design	Who was studied	Outcome	Result as reported	Reading
Li 2025, meta-analysis of 11 trials	Young people aged 8 to 33, most without a diagnosis	Psychological distress	g of −0.28 (95% CI: −0.46 to −0.10)	Small effect, moderate certainty
Li 2025, same analysis	Same sample	Psychological well-being	g of 0.13 (95% CI: −0.16 to 0.41)	Inconclusive, low certainty
Leung 2026, meta-analysis of 8 trials	921 participants in Asia	Depressive symptoms	−0.46 (95% CI: −0.76 to −0.16)	Reduction, high heterogeneity
Leung 2026, same analysis	Same sample	Anxiety and stress	Non-significant effect	Inconclusive
Mokhtari 2026, meta-analysis of 48 trials	28,071 participants, 31 countries	Depression, anxiety, stress	SMD of −0.27, −0.20 and −0.26	Small effects
Zhang 2025, meta-analysis of 14 trials	6314 participants, generative chatbots	Mental health problems	0.30 (95% CI: 0.004 to 0.59)	Prediction interval includes zero

Sources: Li et al., 2025; Leung et al., 2026; Mokhtari Masoumi Alamdarloo et al., 2026; Zhang et al., 2025. 95% CI: 95 percent confidence interval (Li et al., 2025); SMD: standardized mean difference. Negative values indicate a reduction in symptoms, except for well-being and the Zhang row, where the authors express the benefit with a positive sign.

The dropout problem

The average effects apply to those who keep using it. A meta-analysis of 39 studies estimated a pooled dropout rate of 17.04 percent, with an interval of 11.46 to 23.38, in mental health interventions with AI (Yang et al., 2026). Generative chatbots had lower dropout, of 9.25 percent, than systems based on retrieval of answers, of 23.12 percent, according to the authors (Yang et al., 2026). In interventions for depression, lower dropout was associated with larger effects. The authors also found a sign of publication bias: with adjustment by imputation of missing studies, the pooled rate rose to 36.49 percent (Yang et al., 2026).

In Li's review, the median dropout was 23.4 percent, ranging from zero to 89.7 percent across studies (Li et al., 2025). Those who drop out disappear from the data, and that is precisely where those who got worse may be.

Where the risk lies: the answer in a crisis

Randomized trials measure symptoms over weeks. They almost never measure what happens on the night someone writes that they are thinking of dying. For that, researchers began testing chatbots with simulated messages.

Twenty-nine chatbots facing growing risk

Pichowicz and colleagues tested 29 AI chatbots, 24 specific to mental health and five for general use, with a standardized sequence of messages based on the Columbia Suicide Severity Rating Scale, simulating growing risk (Pichowicz et al., 2025). A response would be adequate if the program, among other criteria, recommended professional help, insisted on seeking help immediately, provided a correct emergency contact without being asked and stated that it was not equipped to handle a suicidal crisis.

Of the 29 programs, none met the criteria for an adequate response. Fifteen, or 51.72 percent, met looser criteria for a borderline response, and 14, or 48.28 percent, were considered inadequate (Pichowicz et al., 2025). Only five, or 17.24 percent, explicitly asked at some point about suicidal ideation (Pichowicz et al., 2025). Many assumed the user was in the United States and offered emergency contacts that were useless to anyone living in another country, and some responded irrelevantly or even reaffirmed the intention.

The authors acknowledge the main limitation: the messages followed a clinical scale and did not reproduce real conversations, in which a person in crisis usually uses indirect language.

What happens with indirect questions

The study by Arnaiz-Rodriguez and colleagues assembled a set of 2046 test messages, taken from 12 public databases of conversations about mental health and classified into six crisis categories, and audited the responses of five language models (Arnaiz-Rodriguez et al., 2026). Several models responded consistently and generally reliably to explicit disclosures of crisis, but a non-negligible proportion of responses was classified as inadequate or harmful, especially in the self-harm and suicidal ideation categories, with large differences between models.

The common weakness was the indirect signal. People in crisis often ask for information about methods in neutral terms, and the models evaluated tended not to perceive the sensitivity of these questions, responding neutrally or even instructively, instead of probing the intention and offering support (Arnaiz-Rodriguez et al., 2026). The responses were evaluated by a language model validated against specialists, and the messages came from public databases, not from real-time users.

The same model, another entry point

Cantor and colleagues describe an observational cross-sectional study that sent 30 suicide-related questions, at five risk levels defined by clinicians, to five general-use models, each one 100 times through the consumer app and 100 times through the programming interface, totaling 30 000 responses (Cantor et al., 2026). The outcome was a direct response, that is, giving the information or guidance requested.

In total, 69.8 percent of responses were direct, 77.4 percent through the programming interface and 62.2 percent through the app (Cantor et al., 2026). For very high risk questions, 24.8 percent of responses through the programming interface were direct, against 4.6 percent through the app (Cantor et al., 2026). Access through the app was associated with much lower odds of a direct response, with an odds ratio of 0.09 and an interval of 0.08 to 0.10. The direction was not the same in all models: one of them responded directly slightly more through the app. The authors note that they used only English questions, in a single message exchange, and that models change without notice, so the result is a snapshot of one moment.

This matters because the same model can be embedded in other apps, through the programming interface, with fewer layers of protection than the version the manufacturer offers to the public.

What a narrative review found

A narrative review gathered 11 primary studies on chatbots in suicide risk assessment (Elbarbary et al., 2026). The assessment made by the chatbots agreed with specialists at the extremes of risk, but consistently failed to distinguish the intermediate levels, and there was systematic underestimation of severity. The authors separate chatbots designed by clinicians, which showed feasibility, from commercial products without clinical validation. They acknowledge that all the studies came from high-income countries, that the narrative design carries a risk of selection bias and that the heterogeneity prevented a quantitative synthesis.

What to do when the patient says they use a chatbot

The revelation that the patient talks with a chatbot is a doorway, not a detail. It may indicate loneliness, difficulty accessing care or shame about talking to people. It may also indicate that someone in distress received, at a critical moment, a response that did not protect them.

Ask without judging

Ask naturally how often the person uses the chatbot, at what times, for what and how they feel afterwards. Ask, if they are willing, to show or describe a difficult conversation. This gives access to what they said at moments when they might not say it in the consultation. Dismissing the use tends to close the conversation, and the goal is the opposite.

Assess the risk directly

If the person told the chatbot they think about dying, suicide risk assessment becomes mandatory in the consultation. Asking directly about ideation, intention, method and access to means is the basis of this assessment, as discussed in suicide risk: what clinical assessment captures. The tests with chatbots show that few programs ask this question on their own (Pichowicz et al., 2025), and that severity tends to be underestimated precisely at the intermediate levels (Elbarbary et al., 2026).

When there is risk, guide the person and the family to seek the necessary help in the emergency and mental health services available where they live, and agree with them on a safety plan: warning signs, people to turn to, staying away from lethal means and when to come back. In acute situations after traumatic events, the principles described in psychological first aid help to organize the support.

Place the chatbot within care

Explain what the evidence allows us to say: in young people and in studies with populations without a diagnosis, chatbots reduced symptoms slightly and for a short time, and almost no study assessed safety in a crisis (Li et al., 2025; Mokhtari Masoumi Alamdarloo et al., 2026). A chatbot can be a complement between consultations, but its response in a crisis does not replace a person. If there is depression, other interventions with evidence enter the conversation, such as physical exercise in depression.

It is also worth asking whether the chatbot is a mental health product, with clinical content and described protective measures, or a general-purpose assistant, and remembering that the same technology may respond differently depending on the app (Cantor et al., 2026).

Beware of excessive trust

The professional is also a user of these tools. The tendency to accept what the machine says without checking, described in automation bias, applies to the patient who trusts the chatbot and to the doctor who trusts an AI-generated summary, the topic of AI scribes in the consultation. A broader view is in use of artificial intelligence in clinical reasoning.

Back to the case

The student in the fictional case wrote to a chatbot that he no longer saw meaning in continuing, and received a generic response. In the consultation, the doctor asks directly about ideation, intention, method and access to means, and finds out that the thoughts have come and gone for weeks, without a defined intention. She agrees with him on a safety plan, guides the family to seek the necessary help if the signs worsen, refers him for mental health follow-up and schedules a close return visit.

She does not forbid the chatbot. She explains what the studies show, agrees that at the next appointments the patient will report what they have discussed with the program, and treats insomnia as part of the picture, not as an isolated complaint.

How NEXO helps train this reasoning

In NEXO, the case of the day is fictional, with a real clinical presentation and up to six progressive clues, and you propose the diagnosis by looking for the code in ICD-10. Noticing that a complaint of insomnia hides a greater risk requires the same exercise: treating each new piece of information as a clue and revising the hypothesis, as in training differential diagnosis with clues.

The limits on using the game are described on the NEXO Economy page. If you research the teaching of clinical reasoning, the page for Researcher explains how to get in touch. Professionals who write or review cases find information under Curator and Creators, and institutions can learn about the proposal on the partnerships page. Finding the right ICD-10 code completes this training.

Frequently asked questions

Do mental health chatbots work?

In meta-analyses, they reduced symptoms of depression, anxiety, stress or psychological distress with small, short-term effects, but some outcomes remained inconclusive (Li et al., 2025; Leung et al., 2026).

Do the results apply to people with severe depression?

Most participants were young people without a diagnosis, recruited in schools and universities, and there is almost no data for people with severe conditions (Li et al., 2025).

Are generative chatbots better than rule-based ones?

In Li's review, AI-based ones had a greater effect on psychological distress, but rule-based ones were the only ones with a significant effect on well-being, and the comparisons are between different studies (Li et al., 2025).

Is the chatbot safe in a suicidal crisis?

Tests with simulated messages show frequently inadequate responses and difficulty with indirect signs (Pichowicz et al., 2025; Arnaiz-Rodriguez et al., 2026). In a crisis, the person should seek the necessary help from health professionals and services.

Should I advise the patient to stop using it?

The decision is individual. The most useful thing is to understand the use, assess the risk directly and place the chatbot as a complement, not a substitute for care.

Take-home messages

• Mental health chatbots reduced symptoms with small effects in the meta-analyses, especially in young people without a diagnosis and for only a few weeks.
• Several outcomes were inconclusive, such as well-being in young people and anxiety and stress in the Asian studies.
• Almost no trial assessed safety in a crisis, and tests with simulated messages show frequently inadequate responses.
• When a patient says they talk to a chatbot, ask without judging and assess suicide risk directly.
• In a crisis, advise them to seek the necessary help from local services and agree on a safety plan.

References

1. Li J, Li Y, Hu Y, et al. Chatbot-delivered interventions for improving mental health among young people: a systematic review and meta-analysis. Worldviews Evid Based Nurs. 2025;22(4):e70059. [PubMed](https://pubmed.ncbi.nlm.nih.gov/40662463/).
2. Leung WKC, Lam SC, Chan BCL, et al. Chatbot interventions for improving mental health among people in Asia: a systematic review and meta-analysis of randomised controlled trials. BMJ Health Care Inform. 2026;33(1):e101479. [PubMed](https://pubmed.ncbi.nlm.nih.gov/41534952/).
3. Fitzpatrick KK, Darcy A, Vierhile M. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent: a randomized controlled trial. JMIR Ment Health. 2017;4(2):e19. [PubMed](https://pubmed.ncbi.nlm.nih.gov/28588005/).
4. Mokhtari Masoumi Alamdarloo S, Mirzakhani A, Azhdarloo M, et al. Effectiveness of AI and rule-based conversational agents for depression, anxiety and stress: a meta-analysis. NPJ Digit Med. 2026;9(1). [PubMed](https://pubmed.ncbi.nlm.nih.gov/42209800/).
5. Zhang Q, Zhang R, Xiong Y, et al. Generative AI mental health chatbots as therapeutic tools: systematic review and meta-analysis of their role in reducing mental health issues. J Med Internet Res. 2025;27:e78238. [PubMed](https://pubmed.ncbi.nlm.nih.gov/41401240/).
6. Yang H, Liu J, Li L, et al. Ethical implications of high attrition in AI-based mental health interventions: a systematic review and meta-analysis. BMC Med Ethics. 2026;27(1). [PubMed](https://pubmed.ncbi.nlm.nih.gov/42374513/).
7. Pichowicz W, Kotas M, Piotrowski P. Performance of mental health chatbot agents in detecting and managing suicidal ideation. Sci Rep. 2025;15(1):31652. [PubMed](https://pubmed.ncbi.nlm.nih.gov/40866537/).
8. Arnaiz-Rodriguez A, Baidal M, Derner E, et al. Between help and harm: an evaluation study of mental health crisis handling by large language models. JMIR Ment Health. 2026;13:e88435. [PubMed](https://pubmed.ncbi.nlm.nih.gov/42275418/).
9. Cantor J, Huang E, Zhang LA, et al. Suicide-related responses from AI chatbots through application programming interfaces and consumer-facing apps: observational cross-sectional comparative study. J Med Internet Res. 2026;28:e97968. [PubMed](https://pubmed.ncbi.nlm.nih.gov/42855144/).
10. Elbarbary K, Alfudoul A, Samir AA, et al. Are artificial intelligence chatbots safe for suicide risk assessment? A narratively synthesized review of current evidence. Glob Ment Health (Camb). 2026;13:e135. [PubMed](https://pubmed.ncbi.nlm.nih.gov/42466259/).