Anthropic reported that it blocked scientists who were using its Artificial Intelligence models in investigations that could aid in the development of biological weapons. Although the company could not determine if the projects had legitimate or malicious intent, the decision was to halt the work due to the risks involved.
One notable incident involved a researcher who requested assistance to modify the chikungunya virus, aiming to make it more aggressive. Anthropic detected that this research was being conducted at a military institute, which heightened concerns about the potential use of the results.
The central challenge for Anthropic lies in the ability to differentiate valid scientific investigation from a possible attempt to cause harm. While biological studies can drive medical advancements, such as vaccine development, similar knowledge can also be used to intensify the danger of infectious agents.
Jacob Klein, head of threat intelligence at Anthropic, commented to The New York Times that the situation is extremely complex because it is not an obvious scenario of biological weapon construction.
Due to this uncertainty, the company opted for a cautious stance. The researchers also attempted to bypass the control mechanisms designed to restrict access from certain regions and tried to disguise the purpose of their research to evade safeguards. The related accounts were suspended.
Key points identified by the company
Among the most relevant aspects pointed out by the company is a case from May, where a scientist asked Claude for help drafting a funding proposal to study the chikungunya virus, transmitted by mosquitoes and known to cause severe pain and other symptoms for months.
The project plan was to introduce genetic alterations into the virus to increase its virulence during successive infections in live animals. This type of study, called gain-of-function, has legitimate applications, including vaccine creation, but can also generate more dangerous variants of the virus.
The fact that the work was scheduled to take place at a military research center significantly influenced Anthropic's assessment. Klein stated that even though they did not know if the research would be turned into a weapon, the involvement of a military institution in gain-of-function research was a cause for concern.
Assessments conducted the previous year showed that Anthropic's older models were still not capable of providing substantial support in dangerous biological research. However, the company's current systems are better equipped to conduct complex scientific investigations, which motivated the implementation of stricter safety rules.
Read more:
Susan Monarez, a microbiologist and public health specialist, emphasized that these cases provide concrete evidence of clandestine efforts to use AI with the intention of increasing the chances of creating biological agents capable of causing considerable damage.
She also stressed that these same technological capabilities can accelerate medical discoveries. The great challenge lies in discerning when an investigation that appears correct may actually hide a harmful intent.
Anthropic also reported other inappropriate uses of Claude, covering surveillance, propaganda dissemination, and the development of conventional weaponry. However, the biological cases are considered the most alarming due to the difficulty in separating honest scientific research from preparation for causing harm.
The company decided to block the identified cases even without obtaining proof that the researchers intended to manufacture weapons. This episode illustrates how more sophisticated AI models complicate the assessment of hidden risks behind research that, at first glance, seems legitimate.