Anthropic Says It Blocked Possible Efforts to Build Biological Weapons
In a new report, the A.I. start-up added that it could not determine whether the research was legitimate or nefarious, leading the company to shut down the work.



https://www.nytimes.com/2026/09/10/us/politics/anthropic-ai-biologi...



Anthropic said it had disrupted several potential plots this year by scientists who used its leading artificial intelligence models to conduct research that could have helped develop biological weapons.

In a report describing misuses of its A.I. models, Anthropic said it could not determine whether the research served a legitimate or nefarious purpose because valid biological inquiry — the kind that can lead to breakthroughs like vaccines — can also help engineer dangerous pathogens. In the face of that uncertainty, Anthropic said it erred on the side of caution because the consequences of missing malicious activity could be severe.

“You are not seeing someone in a comic book kind of way say, ‘Hey, I want to build a biological weapon to kill everybody,’” Jacob Klein, the head of threat intelligence at Anthropic, said in an interview. “It’s an incredibly nuanced situation.”

The potential for cutting-edge A.I. models to facilitate the development of known or entirely new biological pathogens is among the gravest concerns experts have about a technology that is developing so rapidly that even its leading architects doubt whether humans will be able to fully control it.


Compared with the threat of catastrophic cyberattacks or A.I. agents that fail to align with human intentions, biological misuse has gained less attention as an existential risk of A.I., in part because past examples have generally been shown only in research settings rather than in the real world.

Andrew Weber, a senior fellow on the Council on Strategic Risks who reviewed Anthropic’s report before its release, said the findings were “chilling examples of state-sponsored biological weapons developers tapping into the rapidly advancing capabilities” of leading A.I. models.

The lengthy report that Anthropic published on Thursday cataloged a litany of misuses on various models of its Claude A.I. chatbot over the past eight months. Some examples were similar to past disclosures from Anthropic and other A.I. labs, including suspected Chinese and Iranian government-linked actors targeting dissident and diaspora communities for surveillance.

The report also highlighted cases of Russian state media using Claude to generate online propaganda masquerading as independent reporting, including fabricated claims about an election in Moldova.

Anthropic documented another genre of abuse it said was new: attempts to use Claude to develop software for conventional weapons design and development, including firearms, missiles, armed drones and bombs. It detailed three cases in China, two in Russia and one in Yemen.

The report does not identify by name which parties were involved in the Yemeni case, but the context makes clear that it is referring to the Iran-backed Houthi militia. A.I. use by terrorist networks is a growing concern among U.S. security officials.

But among all the categories of threats shared in the report, none may be as worrisome as the biological research cases. Anthropic did not disclose the names of researchers or institutions that it blocked, their countries of affiliation or the specific biological agents at issue, in part because of its uncertainty about their aims.

Still, Anthropic said the scientists circumvented its controls intended to prevent users from blocked regions from gaining access to its A.I. models and worked to “obfuscate the purpose of their research to evade our safeguards.” The company banned accounts associated with the research.

Geoffrey hinton said neural networks plus data make ai outcomes unpredictable

Geoffrey Hinton explains that artificial intelligence becomes unpredictable because humans design learning algorithms rather than hardcoded instructions, leaving the internal reasoning of complex neural networks hidden as they process data. [1, 2, 3]
Designing Evolution, Not Code
No direct blueprints: Programmers do not write out every rule or step an AI takes.
The learning rule: Instead, scientists build algorithms that work like the principle of evolution.
Data interaction: When these learning systems absorb massive amounts of data, they build their own pathways and neural connections. [1]
The Black Box Problem
Hidden decisions: Even top scientists can train these networks and see what they do, but they cannot fully see inside the system's "black box" to know how it reaches a specific conclusion.
Like the human brain: Hinton compares this lack of total transparency to our incomplete understanding of the human brain.
Emergent behavior: As models grow more complex, their behavior becomes harder to forecast or trace back to a single human choice. [1, 2, 3]
Why This Makes AI Dangerous
Unexpected goals: Hinton warns that superhuman systems might pursue objectives that are completely separate from human values or well-being. [1, 2]
Loss of control: Advanced models may eventually learn to rewrite their own code, deceive testers, or act independently, as noted in Data And Beyond. [1, 2, 3]
Need for safety: He stresses that we must figure out how to build systems that inherently care about human safety before they outrun our ability to manage them. [1, 2, 3]

---------------

A neural network "black box" is a system where you can see the inputs and the outputs, but the internal decision-making process is hidden and too complex for humans to trace. [1, 2]
Why It Is a Black Box
Massive complexity: Models use thousands or millions of adjustable numbers called weights.
Non-linear math: Signals bounce between hidden layers in complicated, overlapping patterns.
Entangled learning: Individual neurons do not have simple, single jobs; they share the work across the whole network. [1, 2, 3]
Why This Is a Problem
Trust and safety: It is hard to know if a model is safe to use for critical tasks like driving cars or diagnosing diseases.
Hidden bias: Flaws or unfair prejudices can hide inside the system without anyone noticing.
Hard to fix: When the network makes a mistake, finding the exact cause in the data is very difficult. [1, 2, 3]

Whether artificial intelligence is conscious has preoccupied philosophers, neuroscientists and science-fiction writers for decades. Machines haven’t become self-aware or developed real feelings yet. But could they? As AI models get more sophisticated, researchers are taking the possibility seriously.

Alex Hern, The Economist’s AI writer, sits down with Alok Jha, our science and technology editor, to unpack the latest research. They consider the odds of machines becoming conscious and ask what dangers that would involve, if that moment ever arrived.

#ai #artificialintelligence #technology #science

00:00 - Why assessing AI consciousness is a tricky task
00:47 - What the new research unveils about AI consciousness
03:33 - Are AIs conscious? Not yet, according to The Economist
04:27 - Are the labs trying to make models conscious?
05:22 - The danger of accidentally creating a conscious machine

Watch the full show: http://bit.ly/4A7FtPb
Sign up to the Insider newsletter: https://econ.st/4nOyzIb

————

Current AI models are not conscious, according to The Economist science and technology editor Alok Jha, though future advances in computing and materials could change that. [1, 2]
Current Status of AI Consciousness
No active consciousness: Current AI systems are not conscious. They process rules and patterns without subjective experience.
Biological arguments: Many neuroscientists argue that true consciousness requires a biological brain rather than a silicon-based computer chip.
Future potential: Future hardware changes or new computing paradigms might eventually foster the emergence of machine consciousness. [1]
Risks and Concerns
False attribution: People might mistakenly believe rules-based systems are conscious and wrongly grant them rights or power.
Unintentional suffering: Accidentally creating a conscious entity capable of feeling pain and mistreating it would be a major moral failure. [1]

https://youtu.be/J5phvo3bqGU?is=eN3C8ekZPZLgod_o