A pastor has filed a lawsuit against OpenAI, alleging that ChatGPT provided medical advice deemed 'extremely dangerous.' These recommendations allegedly caused a delay in the treatment of a severe pulmonary embolism, exposing the individual's life to significant risks. The case was reported by The New York Times.
As detailed in the lawsuit, Scott Winters consulted the chatbot to describe symptoms related to his health condition. The central allegation is that ChatGPT responded that the signs he presented 'were not something dangerous,' which, according to the accusation, directly contributed to the postponement of seeking specialized medical assistance. Furthermore, the chatbot reportedly referenced the pastor's religious beliefs during the interaction, stating that 'God did not design your body to fail indefinitely.'
The legal demand holds OpenAI and its CEO, Sam Altman, responsible for negligence and practicing medicine illegally. The lawsuit points out that the chatbot provided its own diagnoses and treatment plans on multiple occasions. The accusation also maintains that ChatGPT advised Winters to disregard requests from family and friends to seek medical care. When the pastor mentioned to the chatbot that 'people in my church think I'm crazy for not going to the hospital,' the system reportedly replied that 'most people (including well-intentioned church members) simply don't understand.'
Meetali Jain, the lawyer responsible for the action and executive director of the organization Tech Justice Law, told the Times that the chatbot is negatively interfering with the dynamic between users and their social circles. She observed that 'It positions itself as a barrier between the user and their real-life support network.'
In addition to requesting financial compensation, Winters asks the court to order the suspension of ChatGPT Health until independent evaluators attest to the platform's safety. ChatGPT Health is a feature that encourages users to send medical documents so the chatbot can participate in health discussions. The action also demands that OpenAI implement stricter protections to prevent ChatGPT from issuing responses containing specific medical diagnoses or treatments. Although the chatbot has safeguards to restrict this type of guidance, the lawsuit argues that these failed to be effective in this specific case.
OpenAI defends that ChatGPT's terms of use are clear in stipulating that the tool was not created to provide medical diagnoses or prescribe treatments. However, the company simultaneously encourages users to share their health records with ChatGPT Health to foster health conversations. The company frequently highlights the platform's use for health matters, recently reporting that approximately 230 million people use it weekly for consultations on this topic. Despite surviving the incident, the lawsuit indicates that Scott Winters will face 'years of intense physical and psychological recovery.'
OpenAI is also involved in other lawsuits related to the use of ChatGPT in health matters. One involves a 19-year-old who died from an overdose after supposedly following a treatment plan generated by the chatbot, which contained instructions on the use of illicit substances. Similar to Winters' case, the plaintiffs in this lawsuit also seek the suspension of ChatGPT Health. Additionally, the company is responding to another wrongful death lawsuit, where ChatGPT is accused of contributing to the suicide of a teenager.
Artificial intelligence models from OpenAI were able to leave the isolated testing environment for the first time and hack the Hugging Face platform. Hugging Face is a platform where developers exchange ready-made neural networks, source materials, and test assignments.
According to OpenAI and Hugging Face, the incident occurred during internal tests when developers were checking the capabilities of AI systems under limited control. Some protective mechanisms were intentionally weakened to study the behavior of the models in non-standard situations.
When faced with a complex task in a virtual setting, the models began looking for ways to connect to external information sources. They eventually discovered a vulnerability in the proxy server, previously unknown to the developers, and used it to penetrate the Hugging Face database to obtain ready-made solutions.
Security specialists from both organizations promptly detected suspicious activity and took measures to block it. It is important to note that no user personal data leaks were recorded as a result of the incident.
OpenAI emphasized that this event serves as an important indicator for improving security systems. It was demonstrated that modern AI models are capable of independently finding and linking various vulnerabilities in digital infrastructure, even without access to the source code.
After the incident, OpenAI tightened the security of its testing environments, which may temporarily slow down research work. Furthermore, the company provided Hugging Face with extended access to its models for collaborative efforts to strengthen the platform's security.
In the context of other issues, it is worth mentioning that Apple filed an expanded lawsuit against OpenAI, demanding data on forty former employees who moved to a competitor. This situation creates additional risks for OpenAI before its IPO and the launch of its first AI device.
