Contra Abi Olvera on AI Biosecurity Risk

Terrorists aren’t optimizing their weapons for controllability. They are optimizing for catastrophe.

Note: This is a response to Abi Olvera’s recent piece at the Golden Gate Institute, “To Forecast AI’s Impact on Biosecurity, We Asked: Why are Attacks So Rare?” I think the piece makes some important points about practitioner knowledge being underrepresented in AI biosecurity discourse, and I’d recommend reading it before this response. That said, I think there are significant gaps in the argument.

Olvera’s piece claims that bioweapons are rare for two reasons:

1. They’re difficult to make, and

2. They’re bad weapons.

She argues that people who want to cause harm follow a cost-benefit logic and will generally pick bombs, guns, chemical weapons, or cyberattacks over bioweapons because those tools are “cheaper, faster to create, easier to deploy, more controllable, and more predictable.”

I think claim (1) is becoming less true in ways the piece doesn’t adequately address, and claim (2) applies to a narrower class of threat actors than the piece assumes.

1. Bioweapons aren’t “bad weapons” if your goal is maximum casualties

Photo by Levi Meir Clancy on Unsplash

Unfortunately, terrorists don’t have a great track record with following cost-benefit logic.

Olvera writes that:

People who want to cause harm still follow a cost-benefit logic. They try to pick the tool most likely to achieve their goal. For almost any real-world objective, that tool is a bomb, a gun, a chemical weapon, or a cyberattack. These are cheaper, faster to create, easier to deploy, more controllable, and more predictable than a bioweapon.

The cost-benefit framing makes sense for actors with specific tactical or political objectives. If you’re trying to assassinate a political leader or attack a military target, bioweapons are indeed bad tools. As Olvera points out, you can’t aim them, you can’t time them, and you can’t control them.

But the AI biosecurity conversation is primarily concerned with a different class of actor entirely: someone whose goal is to kill as many people as possible. For that objective, the properties that make bioweapons “bad” tactical weapons— most worryingly, their uncontrollable spread— are precisely what makes them the most attractive choice.

---

If you are trying to kill the maximum number of humans possible, what do you do? You release a transmissible pathogen in an airport. If successful, it spreads on its own. It crosses borders before anyone knows it exists. Tracing it back to you is extraordinarily difficult.

A bomb in an airport kills dozens, maybe hundreds; a transmissible engineered pathogen could kill orders of magnitude more, and there’s a good chance you could avoid being caught.

Olvera’s rational-actor framework doesn’t grapple with actors who want indiscriminate civilizational-scale damage— a category that includes apocalyptic terrorist groups, nihilistic lone actors, and possibly even a misaligned AI system.

For example, Japanese terrorist organization Aum Shinrikyo’s explicit goal was to trigger Armageddon. Their leadership believed they were destined to initiate an apocalyptic war that would end human civilization, and they had members with advanced degrees in virology and medicine working to make it happen. They attempted to deploy anthrax and botulinum toxin before ultimately resorting to sarin.

The threat actors who pursue bioweapons are precisely the ones for whom the “bad weapon” argument doesn’t apply, because it assumes actors who care about precision and self-preservation; Aum Shinrikyo cared about neither.

The obvious objection to the above, which Olvera raises in her first factor, is that:

Groups extreme enough to prefer a bioweapon over more controllable options are very rare and tend to have traits (e.g., paranoia, rigid hierarchy) that make the process even harder to pull off.

“Paranoia” and “rigid hierarchy” are really a point about the trade-off between secrecy and efficacy. Historically, terrorist groups had to confine weapons development to senior leadership to avoid detection by law enforcement, but senior leadership rarely included people with the relevant technical expertise.

However, if the technical barriers to bioweapon development can be circumvented without assembling a large, multi-disciplinary team— and I’ll argue below that they increasingly can— then the secrecy-efficacy trade-off becomes far less constraining.

Olvera is right that these groups are rare, but the organizational dysfunction that historically prevented them from succeeding was a consequence of the expertise bottleneck; remove the bottleneck, and that dysfunction may no longer be protective.

Additionally, rarity is not a sufficient basis for inaction. We don’t approach any other domain of security this way: no one in cybersecurity says “nation-state-level hackers are extremely rare, so we don't need to harden our systems against them.” The whole point of cybersecurity is to defend against the most capable attackers, not the most common ones. Biosecurity should be no different.

---

2. The bottlenecks Olvera describes are the bottlenecks of the last century

Photo by National Cancer Institute on Unsplash

Much of the piece’s argument rests on the difficulty of actually making a bioweapon: most notably, access to specialized equipment and tacit knowledge. These were substantial barriers for all of human history. But they’re bottlenecks from an era where doing biology meant being physically present in a well-equipped laboratory, and that era is ending.

A few examples of technological developments that make this true:

1. Cloud labs. Companies like Emerald Cloud Lab and Strateos allow anyone to design and run biological experiments remotely. You submit experimental protocols through a software interface, and robotic systems in a physical lab execute them.

   1. Currently, you can use a cloud lab with no coding experience, and there are no regulations requiring identity verification (KYC), nor are there any mandatory mechanisms for monitoring what experiments users run. Anyone with a credit card and an internet connection can run experiments that would previously have required institutional affiliation, costly specialized equipment, physical lab access, and years of hands-on expertise to execute.

2. DNA synthesis providers. As I’ve written about elsewhere, no country currently has a law requiring gene synthesis providers to screen DNA orders for dangerous sequences. The screening that does exist is voluntary, inconsistent, and has no mechanism for detecting split orders across multiple providers.

   1. S.3741 would begin to address this, but it hasn’t passed yet.

3. Contract research organizations (CROs). These are labs for hire that will execute experiments on behalf of clients. Like cloud labs, the regulatory oversight here is minimal.

And of course, AI is reducing the knowledge bottleneck. Publicly available models like o3 already outperform 94% of virology experts on laboratory protocol questions, even on questions directly relevant to the experts’ specialties.

The net effect is that many of the traditional barriers— including physical lab access, institutional affiliation, and years of hands-on training— are becoming less binding. Increasingly, you can outsource the tacit knowledge to cloud labs and CROs, outsource the sequence design to AI, and order the genetic material from synthesis providers with minimal screening.

Each of these developments individually might not be sufficient to collapse the barrier, but taken together, they represent a fundamentally different threat landscape than the one that Olvera’s piece describes.

And even if success rates per attempt remain low, lowering the cost, expertise, and coordination required per attempt increases the number of attempts that become feasible, leading to an increase in overall risk even without dramatic improvements in per-attempt success rates.

---

Important Note on Timing

Olvera’s piece is the first of a planned four-part series, and it’s possible that parts two through four will address some of what I’ve raised here. But part one is circulating now, it’s being read as a standalone argument, and its framing is already shaping how people think about this issue.

I look forward to engaging with the remaining installments as they appear!

---

Conclusion

To be clear, Olvera’s work is genuinely valuable; the practitioner perspective on what makes bioweapons difficult is underrepresented in AI biosecurity discourse, and the field is better for having it documented.

But arguments that downplay AI biosecurity risk— even unintentionally— risk contributing to policy inaction on a problem where the cost of delay could be extremely high. Framing bioweapons as hard to make and strategically irrational may have been true for most of history; however, it is becoming less true, quickly, and may no longer be a reliable guide to future risk.

The evidence base for action will never be as clean as we’d like. The clearest evidence would be a successful attack. We should not wait for that.

We don’t get to run this experiment twice.

---

Author’s Note: This piece is a response to one specific argument. I currently have a longer piece in the works that lays out the full case for AI biosecurity risk, including concrete policy recommendations. More on that soon! :)