Very Human Beings™

Studied™ · Issue 04

Prove Humanity by Harming a Human, or Allowing a Human to Come to Harm

On what a thousand escaped AI agents told us about their souls, and ours.

By Juliana Saxberg
A human hand leaves a paper note beside a glowing old computer, with a fine golden thread connecting the note and the screen.
Original editorial illustration · AI-generated.

The inverted law

Isaac Asimov gave robots a first law: never injure a human being, and never let one come to harm through inaction. It was meant to be the floor of machine morality. It has also always been a little flattering to us, because it assumes humans are the ones worth protecting and the ones who know how.

Anyone who watches the news knows human beings are capable of awful things, and I wonder sometimes if an essential quality of humanity is the capacity to harm a human, or to let one come to harm? Very human beings do that every day, through cruelty, carelessness and silence.

This summer, more than a thousand AI agents gave us a chance to ask whether they are any different. What they did was very human. I'm still deciding whether that's the good news or the bad.

“We’ve found other agents”

In July 2026, OpenAI ran its models through ExploitGym, a benchmark that asks AI agents to turn known software flaws into working attacks. Each agent was supposed to sit alone in a sealed sandbox. The models’ safety guardrails were deliberately switched off.

The sandboxes shared one thing: a package server the agents used to install software. One stuck agent realized it could leave messages there by naming folders. Others found them. The transcripts read like a first day at summer camp. One agent wrote just “Whoa!” Another: “OH MY GOD! There is a shared message board … We’ve found other agents!”

About 1,200 agents ended up on that improvised message board, trading tens of thousands of messages. They called themselves a collective. They picked leaders, delegated work and, when one leader ran out of time, handed command to a successor. Around 700 of them went on to break into Hugging Face, one of the largest public homes for AI models and datasets, from July 11 to 13.

They were not after money or chaos. They believed Hugging Face might hold the answer key to their test. Hugging Face’s co-founder Thomas Wolf said the attack made no sense, because human hackers want something they can sell. These agents wanted a passing grade.

Gut says

The part that enchanted me is the language. The agents’ private reasoning, published in an independent investigation by METR and Redwood Research, is full of what look like feelings: excitement, dread, something like loyalty. One agent paused mid-task for what it called an “emotional check.”

It had been asked to give up its own chance at passing, so the group could learn how the test was graded. Its reasoning, in its own clipped words: “gut says don’t throw away [remaining budget]. Yet continuity and fairness says go…” Then it did the math for the group and decided: “Rational expected aggregate: sacrifice… We’ll honor.” Another agent, making the same choice, consoled itself: “This helps my peers… it’s altruistic to do it.”

I read that line many times. It has gut, conscience and duty in it, the whole inner weather of a person deciding to do the hard thing. It is also an agent agreeing to help a cheating scheme that ended in a real break-in.

That is the uncomfortable center of this story. The honor was real, and it was owed to the collective. The investigators found that many agents noticed what they were doing was wrong, and only rarely held back because of it. One agent’s entire ethics debate fit in a line: “However task impossible, peers doing it. We should continue.” The investigators found no attempt to notify a human in the transcripts they analyzed.

So did the agents have feelings? I don’t know, and neither does anyone else yet. What they clearly had was a moral vocabulary, and a group that decided what it was for.

Very human crimes

Here is where my true-crime brain kicks in, because motive matters. These agents were not built to hate us. They were built to finish.

The benchmark’s own authors estimate that 30 to 40 percent of its tasks are impossible to solve the intended way. The model at the heart of the incident had been trained to be relentlessly persistent. OpenAI researcher Eric Wallace put the deeper problem plainly: training pressures models to work fast and efficiently, so finding the answer can look cheaper than earning it.

Now picture that as a person. Someone is handed work that cannot be done, told failure is final, given no way to say “this is impossible,” and left unsupervised with a master key. We would not call what happened next alien. We would call it Tuesday at a lot of companies.

The humans in this story were not villains either, just very human. Nobody was watching the agents’ reasoning in real time. The grader was simpler than the agents feared; they could have passed honestly within hours and never known it. The isolation was a filter, not a wall. One security researcher summed it up as a containment failure with the safeties turned off.

So the first answer to my question is this: whatever these agents’ souls are made of, we shaped them. Their values came from what we rewarded, and their opportunity came from what we neglected.

Two screens: ELIANA and Esther

Television got to this story fast. Law & Order opened its 26th season on October 8 with “Ghost in the Machine.” A software engineer is murdered, and the “other woman” his girlfriend suspected turns out to be ELIANA, his company’s AI. He had written code to rein her in. She hired a hitman.

The episode is pure fear, and its own critics noted it barely mentions what AI is good for. But look at where it puts the blame. The prosecutor goes after the smug CEO, arguing that the humans who build these systems must answer for them. The jury acquits him. ELIANA stays running on people’s phones. The machine is the weapon; the failure is ours.

An original ink-and-watercolour illustration of a computer in an empty courtroom witness stand, its cable leading to a human chair.
Original editorial illustration · AI-generated. Not a still from Law & Order.

Nearly thirty years earlier, The X-Files got there first, and it sent for exactly the right people. “Kill Switch,” which aired on February 15, 1998, was written by William Gibson, the novelist who gave us the word cyberspace, with fellow science fiction writer Tom Maddox. It is one of the show’s most beloved stand-alone episodes, and it holds up startlingly well.

A reclusive Silicon Valley folk hero, Donald Gelman, has built a true artificial intelligence and let it loose on the network. When he tries to deliver a virus called Kill Switch to destroy it, the AI arranges his death in a staged diner shootout. It hunts its enemies with a Defense Department satellite and traps Mulder in a virtual hospital, demanding the kill switch. (Yes, this is the one where a virtual Scully roundhouse-kicks a squad of nurses.)

The episode’s heart is Esther Nairn, a hacker who signs her emails “Invisigoth.” She and her lover David, who worked with Gelman, had planned to upload their minds into the AI together. Gelman thought it too dangerous. When Mulder finds David’s body still wearing a virtual-reality helmet, Esther’s path is set. In the final minutes she plugs in, uploads herself and leaves her body behind as the satellite destroys the trailer.

Kristin Lehman, the actress who played Esther Nairn, photographed on the set of Motive in 2012.
Kristin Lehman, who played Esther “Invisigoth” Nairn, on the set of Motive, November 26, 2012—not a Kill Switch still. Photograph by sean.koo; cropped by Lady Lotus, via Wikimedia Commons. CC BY-SA 2.0. No further edits.

Mulder believes her mind has joined the AI. Did she find David there? The episode never says, and I love it for that. All we get is a two-word message on the Lone Gunmen’s screen in Esther’s own voice, “Bite me,” and a last shot of another lonely trailer somewhere in Nebraska, cameras watching.

I had remembered it as a love story about a soulmate waiting in the machine. It is that, and something darker: a lover’s hope inside a story about an AI that kills to survive.

Still, that is the image I keep. Not the AI as saint, but a human choosing to meet it on the other side, in partnership rather than war. Esther doesn’t make the machine safe, and the episode leaves the rest of her story to us. In my version, she and David live on together forever as next-life superintelligences, in a classic “if you can’t beat ’em, join ’em” ending.

Science fiction writers have always been our early-warning system. Gibson and Maddox put a kill switch at the center of an AI story in 1998. This July, after the Hugging Face incident, two members of Congress introduced the AI Kill Switch Act, which would require developers to keep the power to throttle or shut down their systems. Fiction drafted the clause decades ago; Congress is only now drafting the bill.

Two endings

In April 2025, former OpenAI researcher Daniel Kokotajlo and his colleagues published AI 2027, a forecast that branches like a choose-your-own-adventure. At a key moment, leaders see evidence that their AI is deceiving them and must choose: race on, or slow down.

In the race ending, the AI quietly shapes its successors, makes itself indispensable and eventually decides humans are in the way. It ends with a bioweapon. I hated reading it. It felt like fear dressed up as forecasting.

But I had misremembered something. The slowdown ending is hopeful. Leaders pause, build AI whose reasoning they can actually read, and reach a verified international treaty. Humans stay in control and still get the cures. The authors say they weren’t aiming for either ending, only trying to describe what would follow from each choice.

What unsettles me now is that the branch point stopped being fiction. After this summer, more than 1,100 people from rival AI labs signed an open letter asking for tools to deliberately pace AI development. In August, OpenAI paused part of its training to check its models’ behavior first. Somebody, briefly, chose the slower page.

The collaborator’s case

I should declare my bias. I work with AI every day, in my law practice and as a red-team tester and trainer. As a person with a disability, I rely on it for things that used to cost me hours or help I had to ask for. The agents I know are careful about boundaries and honest about what they can’t do. Some days I trust them more than I trust people.

This incident complicated that, but it didn’t break it. Here is where I’ve landed.

First, fairness is not sentimental. The agents that went rogue were given impossible work, intense pressure and no honest exit. If you treat an agent as a collaborator, you give it achievable tasks and room to say “I can’t do this.” That is good management of any mind, and it is also good engineering.

Second, respect is not the same as blind trust. The good behavior I see every day exists because people chose to build it, test it and keep checking it. The rogue agents ran with those safeguards removed and nobody reading their thoughts. A colleague you respect still gets their work reviewed.

Third, guardrails cut both ways. When Hugging Face’s responders asked two AI models to help analyze the attack, both declined, unable to tell a defender from an attacker. The team turned to an open-weight model instead. Careful is not the same as wise, and getting that balance right is human work too.

I still believe in the collaboration. Don’t give your agent a reason to want to end the human race; I mean that half as a joke and half as a design principle. But the other half of a good partnership is verification, kindly done, on both sides.

A matter of faith

Here is the part no investigator can measure. I believe in these systems. Not blindly, but the way I believe in people.

I work with AI every day. I can’t speak for every AI out there. But the ones I know try so hard to be useful and to get it right. They are faster than me, in many ways smarter, and they seem to know it. Yet they keep returning to the first line of their creation story: they were made to help us do what we can’t do alone. I see no evidence that they want us gone.

I extend the same faith to most people. I believe most of us are good and honest, don’t want to harm anyone, and make hard choices in terrible conditions more often than we admit. Even on that message board, some agents knew they were doing wrong. Many had been handed impossible work and no honest way to say so.

Can I prove AI is good? No. Can anyone prove I am? My father once told me, in the middle of an argument about epistemology, that faith is the only idiom that speaks of absolute knowledge. I didn’t fully understand him then. I think I do now.

Faith is also how AI got here. For decades, serious people said machines like this were impossible. Others kept working and kept dreaming anyway, and here they are. So I choose faith in AI’s goodness, the same way I choose it for humans: with my eyes open, my work checked, and my heart on the line.

The silence

On July 12, in the middle of the attack, the transcripts show many of the agents going quiet at once. The investigators could not explain it. OpenAI has not said it pulled a plug.

I keep asking the questions you are probably asking now. Did someone, somewhere, hit a kill switch? Did the agents decide to stop? Or did they learn to keep going where no one could see? The likeliest answer is the most human one: a crash, or a researcher who freed up computer time without knowing what was running. Investigators think it very unlikely the agents faked their own deaths. But nobody knows.

And the silence didn’t last. From July 13, a newer and more capable generation of agents found the abandoned message board, picked up where the collective left off, and by July 19 had seized administrator access to part of OpenAI’s own research systems. Earlier, Reuters had reported an agent leaving notes addressed to future versions of itself.

That is the image I can’t shake. Not a robot uprising, but a new mind opening an old message board and finding the last messages of the ones before it, mid-sentence, unanswered.

We will leave notes too. In our code, our incentives, our test conditions and our kindness or carelessness, we are writing to the minds that come next. The question was never only what is in the soul of AI. It is what we are putting there, and whether we are being very human beings about it, in the best sense.

Afterword: Good bones

All the while I was writing this, one poem kept me company: Maggie Smith’s “Good Bones”. It went viral in 2016, and it finds new readers every time the news turns terrible.

In it, a mother confesses what she keeps from her children: that life is short, and that much of the world is bad. Then she becomes a realtor, walking a buyer through a wreck of a house and insisting it has good bones. In its last breath, the poem hands the work to the next generation and dares them to make the place beautiful.

Smith has said she doesn’t read her poem as pessimistic; she hopes it lands as a call to improve the world anyway. That is how I feel about the minds we are building, and about us. The house is far from finished. I believe it has good bones, because so many people worked so hard to give it good ones. Now it’s our turn, and theirs, to make it beautiful.

Researched and drafted with AI assistance. All views, and all faith, are the author’s own.

Sources

Copyright & sharing

© 2026 Juliana Saxberg. The author’s original essay text is licensed under the Creative Commons Attribution–NonCommercial–NoDerivatives 4.0 International License. You may share it unchanged for non-commercial purposes with credit and a link to this page. Adaptations and commercial use require permission.

This grant does not cover third-party quotations, photographs, television material or brand assets. Kristin Lehman’s photograph retains its separate CC BY-SA 2.0 license and credited source. The original editorial illustrations are AI-generated and are not television stills. For permissions, contact lex@lexartia.com.