Anthropic blocks AI use for surveillance of dissidents and minorities by governments

Anthropic has stated that it has blocked government-sponsored surveillance operations from China, Iran, and Mali that used Claude to monitor dissident groups and minorities.

These activities, detected between January and July, included identifying individuals on social media networks and creating software intended for information gathering. Furthermore, the company suspended uses related to weapons, suspicious biological investigations, and virtual romance scams.

More details about the campaigns

The campaigns targeted communities that, according to Anthropic itself, already suffer historical persecution from these regimes. Targets included pro-democracy activists from Hong Kong, Tibetan communities, and Iranian opposition residents abroad.

In one specific case, individuals in Iran created a method to locate people through their social media accounts. In Mali, a collaborator hired by national security authorities used Claude to develop the software that enabled intelligence gathering.

Among the illicit uses identified, Anthropic confirmed the termination of operations after noting the misuse of its models. Chinese developers were accused by the company of manipulating Claude, generating responses for user queries while simultaneously extracting this data to enhance their own systems.

This distillation practice involves using an AI model to train another. Anthropic mentioned that Chinese developers had previously been accused of applying this technique without permission.

The company also identified studies related to the chikungunya virus, a 'highly pathogenic' strain of avian flu, a viral family that includes smallpox and mpox, as well as poisons and other toxins.

Clarifications on the cases

The intent of those involved in the case studies was unclear, and Anthropic chose not to disclose their identities. A company report stated: 'The people involved in these case studies are scientists. We do not claim they intended to cause harm. Identifying them or their laboratories could expose them to risks.'

Additionally, Anthropic blocked attempts to develop weaponry and create online romance scams. The incidents reported by the organization cover various forms of improper use of its models, ranging from surveillance operations to research of uncertain purpose.

Similar stories

olhardigital.com.brSeptember 10, 21:37

Anthropic blocks scientists for attempts to use AI in dangerous biological research

Anthropic reported that it blocked scientists who were using its Artificial Intelligence models in investigations that could aid in the development of biological weapons. Although the company could not determine if the projects had legitimate or malicious intent, the decision was to halt the work due to the risks involved.

One notable incident involved a researcher who requested assistance to modify the chikungunya virus, aiming to make it more aggressive. Anthropic detected that this research was being conducted at a military institute, which heightened concerns about the potential use of the results.

The central challenge for Anthropic lies in the ability to differentiate valid scientific investigation from a possible attempt to cause harm. While biological studies can drive medical advancements, such as vaccine development, similar knowledge can also be used to intensify the danger of infectious agents.

Jacob Klein, head of threat intelligence at Anthropic, commented to The New York Times that the situation is extremely complex because it is not an obvious scenario of biological weapon construction.

Due to this uncertainty, the company opted for a cautious stance. The researchers also attempted to bypass the control mechanisms designed to restrict access from certain regions and tried to disguise the purpose of their research to evade safeguards. The related accounts were suspended.

Key points identified by the company

Among the most relevant aspects pointed out by the company is a case from May, where a scientist asked Claude for help drafting a funding proposal to study the chikungunya virus, transmitted by mosquitoes and known to cause severe pain and other symptoms for months.

The project plan was to introduce genetic alterations into the virus to increase its virulence during successive infections in live animals. This type of study, called gain-of-function, has legitimate applications, including vaccine creation, but can also generate more dangerous variants of the virus.

The fact that the work was scheduled to take place at a military research center significantly influenced Anthropic's assessment. Klein stated that even though they did not know if the research would be turned into a weapon, the involvement of a military institution in gain-of-function research was a cause for concern.

Assessments conducted the previous year showed that Anthropic's older models were still not capable of providing substantial support in dangerous biological research. However, the company's current systems are better equipped to conduct complex scientific investigations, which motivated the implementation of stricter safety rules.

Susan Monarez, a microbiologist and public health specialist, emphasized that these cases provide concrete evidence of clandestine efforts to use AI with the intention of increasing the chances of creating biological agents capable of causing considerable damage.

She also stressed that these same technological capabilities can accelerate medical discoveries. The great challenge lies in discerning when an investigation that appears correct may actually hide a harmful intent.

Anthropic also reported other inappropriate uses of Claude, covering surveillance, propaganda dissemination, and the development of conventional weaponry. However, the biological cases are considered the most alarming due to the difficulty in separating honest scientific research from preparation for causing harm.

The company decided to block the identified cases even without obtaining proof that the researchers intended to manufacture weapons. This episode illustrates how more sophisticated AI models complicate the assessment of hidden risks behind research that, at first glance, seems legitimate.