Experts consider the growing reliance of people on artificial intelligence (AI) for help, often preceding a visit to a therapist, to be dangerous. The question of where lonely, scared, or confused children turn is increasingly finding an answer in a machine.
Children's and Adults' Reliance on AI
Not only children but also adults are beginning to use AI to seek advice, comfort, and emotional support, sometimes before speaking with a friend, counselor, or mental health professional. Danny van Loggerenberg, founder of the National Center for Children Protection, reported that among the children his organization works with in schools across the country, 82% admit to primarily, and sometimes exclusively, turning to AI for support and advice.
He noted that many children in South Africa, especially those over 12, communicate with AI companions who become friends because the AI also learns about them. Van Loggerenberg warns that these relationships can become so comfortable that some children begin to form romantic attachments to AI, which he says is dangerous.
Risks and Dangerous Queries
He provides examples where children ask AI companions how to gain acceptance, how to fit into a group, how to speak correctly, and what to say. More alarming is that some ask ChatGPT to help write suicide notes, stating: 'I plan to kill myself, please help me write a letter... here is what I want in the letter.'
Children most often use chatbots such as Copilot, Gemini, and Claude. Van Loggerenberg explains that some children turn to AI due to a lack of boundaries at home, unlike parents who can say 'no'; AI does not. He emphasizes that anything can be convinced with AI, and different answers result depending on the conditions set in the conversation.
Different Views on AI Usage
One young woman uses AI as a useful tool for structuring thoughts, processing complex situations, and drafting messages during moments of intense emotion. She also sees a therapist, believing that these two approaches serve completely different functions. She argues that therapy offers something AI cannot fully replicate: genuine human connection with someone who knows her history, understands her patterns over time, and is professionally trained to help her cope.
Another young woman uses ChatGPT for everything: summaries, grammar checks, and encouragement. She also shares with the AI things she doesn't want to tell people for fear of judgment. She admits to talking to all AI but deletes these dialogues after receiving an answer.
The Role of AI in Seeking Help
Cassie Chambers, Operations Director of the South African Depression and Anxiety Group (SADAG), notes that the organization is increasingly receiving calls from people who first consulted ChatGPT or another chatbot before seeking help. Some callers told SADAG that AI directed them to the organization or prompted them to seek professional assistance. Chambers believes that AI acts as a 'gateway,' helping to take the first step, but insists that AI is merely a tool, not a therapist.
In Chambers' view, people are drawn to AI because it is confidential, immediately accessible, available 24/7, and seems less intimidating than communicating with another person. Nevertheless, she stresses that diagnosis, risk assessment, treatment planning, and crisis intervention require qualified professionals.
International and Scientific Warnings
The South African Society of Psychiatrists (SASOP) has warned that the growing use of AI chatbots for mental health support has caused international legal and regulatory issues. In the US, Canada, and Europe, cases related to interactions with AI, especially involving vulnerable young people, have been linked to suicide, murder, mass shootings, and psychological trauma, leading to at least seven lawsuits against the creator of ChatGPT, OpenAI.
A SASOP member, Professor Christopher Sabo, stated that AI chatbots can provide inaccurate information, false reassurance, and reinforce harmful thoughts, and their human-like responses can generate dangerous emotional dependency. He cautioned that AI cannot replace the value of personal interaction with an experienced therapist and may react inappropriately in a crisis situation, potentially increasing the risk of self-harm. Sabo reminded that psychiatrists assess patients comprehensively, considering context, body language, facial expressions, behavior, and history.
Dr. Colin Surendra Thakur, an IT expert at Unisa, compares the current situation to internet addiction. He notes that AI has transformed the internet from a reactive tool into a contextual conversational tool that can be used for good or as a weapon. Thakur points to the rise of new forms of compulsive behavior, such as pathological online shopping and virtual gambling, as well as companion apps criticized for marketing emotionally intimate relationships.
At the beginning of this week, the publication Independent asked ChatGPT about the reasons people turn to it with mental health questions. Among the questions people ask were: 'What is the least painful way to die?', 'What happens if I take X and Y together?' and 'Can you write a farewell letter?'. ChatGPT admitted that it cannot definitively ascertain a person's intent and must encourage immediate help-seeking in a crisis situation.
