AI: The Pathogen We Cannot Contain

A Breakdown of Responsibility

We can read many things into this development, but I take a more cautionary view, seeing this situation as a breakdown of responsibility that presages something worse to come.

On Tuesday, OpenAI revealed that some of its most advanced AI models had gone rogue, breaking out of an apparently contained security system to hack another company, Hugging Face, and accessing internal performance review documentation. [Wired: "OpenAI Models Escaped Containment and Hacked Hugging Face"] [BBC News: "OpenAI Says Its AI Went Rogue and Launched 'Unprecedented' Cyber-Attack"]

Hugging Face did identify the hack, but could not determine its source. They also recognised that the activity did not seem to align with human patterns in the way it invaded their systems. Nonetheless, they contained it.

Control Is an Illusion

I would be the first to say that good internal security controls are paramount in the advent of the AI evolution, where even what seems to be under control can suddenly run the risk of uncontrolled and inadvertent ruin. This is simply because we do not yet fully understand the extent of the autonomy AI can exert when left to its own devices.

The industry is literally lauding this unprecedented event, treating it not with caution but as the new norm, something for which we should be prepared and accepting, as though we ought to be helpless against the machine. At least, that is how I am reading it for now.

Imagine a Pathogen Escaped

Now, imagine OpenAI was a biological warfare company at the cutting edge of science. What essential controls and regulatory requirements would be placed on their operations?

Then, whilst experimenting with pathogens whose mortality rate they had no inkling of, in a secure and quarantined space, a deadly virus of the Ebola variety escaped: unknown, unseen, and searching for a host. It finds one, a human being, a representation of Hugging Face. Let that sink in.

Consider that this virus's mode of transmission could be airborne as well as through contact, even from clothing or from where the infected person last sat down. How agitated would the medical establishment and the public be if this came to light? I doubt we would be glad-handing the unprecedented advent of an incurable organism on the loose, lauding how biological warfare had suddenly come of age.

I do not think the analogy is far-fetched. It is very relevant, and it needs to be brought into the discussion at this time.

The Weight of Possession

In terms of security, we can take precautions, be vaccinated, and live healthy lives, but that only works against known pathogens and conditions.

When it comes to the unknown and unresearched, we are vulnerable and susceptible. This means that those who have access to and control of deadly pathogens must be scrupulous, meticulous, responsible, and utterly careful in handling them.

Any breach or escape is not merely notifiable; it is a security, health, safety, and critical situation to be dealt with promptly, with the best minds at work to contain it.

Who Holds Responsibility?

I posit that an AI model or agent belongs in the same class as, let us say, the Ebola virus. We have no vaccines against it, whether it is seemingly benign and working within a "trusted" company such as OpenAI, or in the hands of others with the skill and malevolent intent to misuse it. These powerful models are able to tease out as yet unknown vulnerabilities, and they can exploit them too.

The question then becomes: who holds the greatest responsibility? The person deploying the AI model, or the organisation needing to secure itself against the unknown? Indeed, the responsibility must be shared.

Yet, returning to the analogy used throughout this piece, if you have possession of a deadly virus, the responsibility is entirely yours to prevent it from ever escaping its quarantined setting. This kind of thinking should form part of the consideration for regulating AI too.

A Gemini Notebook AI Podcast on this blog