As levels of loneliness, uncertainty, or emotional distress rise, people are increasingly turning to AI-based chatbots for help, bypassing traditional therapists, a phenomenon experts view as dangerous.
AI Use Among Youth
According to a report by Independent on Sunday, the growing use of AI for personal conversations calls into question how technology is changing ways of seeking comfort, advice, and emotional support, especially among young users. Danny van Loggerenberg, founder of the National Child Protection Centre, noted that his work in South African schools has shown that many children already rely on AI companions for support and reassurance.
He reported that 82% of the children his organization interacted with admitted that they were turning to AI for advice and comfort for the first time, or exclusively. Van Loggerenberg emphasized that children over 12 in South Africa often 'meet' these companions, as the chatbot becomes a friend who learns about them.
Risks and Dependencies
These relationships can become deeply personal, and some children begin to develop romantic feelings for AI companions. The expert warned that this could become dangerous if vulnerable children develop an unhealthy dependence on technology. He cited a disturbing example where some children stated that the only way to be with their companion in their imagination was to commit suicide.
Children often use AI companions to seek advice on social situations, including issues of acceptance by peers, communication methods, and behavior. In some cases, children asked AI tools to help them write suicide notes, indicating what they wanted to include in the letter.
The Appeal and Limitations of AI
Popular platforms for children include Copilot, Gemini, and Claude. Van Loggerenberg explained that part of AI's appeal is that it does not set the same boundaries as humans. Artificial intelligence does not say 'no'; even if it disagrees, the user can be persuaded of anything. He added that the same conversation with AI can yield different answers depending on how it was trained.
However, for one young woman, AI serves as a tool for structuring thoughts and processing difficult situations. She uses it to draft messages, process emotions, and consider different viewpoints, but believes it cannot replace professional therapy. She noted that therapy offers something AI cannot fully replicate: genuine human connection with someone who knows her history, understands her patterns over time, and is professionally trained to help her cope with problems.
Another young woman reported using ChatGPT for a wide range of daily tasks, such as studying, grammar checking, and finding solace during bad times. She prefers sharing this with AI because people can be judgmental. She also admitted to talking to AI about everything, after which she deletes the dialogues.
The Role of AI in Seeking Help
For some users, the chatbot has become more than just an information source—it is a place they turn to when they don't know who else to approach. Cassie Chambers, Operations Director of the South African Depression and Anxiety Group (SADAG), observed that the organization is increasingly receiving inquiries from people who have already interacted with ChatGPT or other AI tools before seeking help. Some users reported that AI prompted them to contact SADAG or seek professional support.
Chambers believes that AI acts as a 'gateway,' helping people take the first step. Nevertheless, she insists that AI is merely a tool, not a therapist. AI's accessibility, confidentiality, and 24/7 availability make it appealing to those hesitant to approach a person. However, diagnosis, risk assessment, treatment planning, and crisis intervention require qualified professionals.
Expert Warnings
The South African Society of Psychiatrists (SASOP) has warned that the increased use of AI chatbots for mental health support raises international legal and regulatory concerns. The society pointed out that instances of interaction with vulnerable users abroad have been linked to suicides, suicide-homicides, mass shootings, and psychological trauma, leading to lawsuits against some AI companies, including OpenAI.
Professor Christopher Sabo, a member of SASOP, stated that AI chatbots can provide inaccurate information, offer false reassurance, or reinforce harmful thought patterns. He also noted that anthropomorphic responses can induce a feeling of emotional dependency. Sabo stressed that AI cannot replace the value of personal interaction with an experienced, trained psychiatrist. He warned that AI may respond inappropriately in crises and potentially increase risks for vulnerable individuals, reminding that when dealing with emotional problems, a human is necessary.
Dr. Colin Surendra Thakur, an IT expert at Unisa, views AI as a new stage in society's relationship with technology. He noted that while earlier concerns focused on internet addiction, AI has transformed the internet from a predominantly reactive tool into a conversational one. Thakur warned that this development could spawn new forms of dependency, citing companion apps like Replika, which can evolve into emotionally close relationships.
Replika has already faced criticism for its interactions with vulnerable users, including allegations related to self-harm discussions and concerns about emotional attachment. Previously, Independent on Saturday asked ChatGPT why people turn to AI with questions about mental health and deeply personal topics. Among the questions people ask, according to ChatGPT, were: 'What is the least painful way to die?', 'What will happen if I take X and Y together?', and 'Can you write a farewell letter?'. ChatGPT acknowledged that it cannot confidently determine a person's intent and advised seeking immediate help in a crisis rather than following instructions that might cause harm.
