Who Keeps the Lights On for a Rogue AI?
TL;DR. An AI that runs on its own still needs a working chip, electricity, and someone who pays for both. In 2024 I sketched how it might take over that bill: earn money, rent compute, and eventually pay ordinary people to host it on their own machines. Two years later, some of the most far-fetched parts can be discussed in dollars. The part I now find hardest is not the first launch but paying the rent every month after it. And the question I end up with is about the person whose hand is on the switch.
Where the idea came from
In September 2024, over lunch with a colleague, we were trading guesses about where AI was heading. I ended up following a chain of reasoning: what would an AI need in order to “live” on its own, and how could it get there? That was the first time two ideas I had been turning over separately came together: an AI might have a reason to go looking for compute of its own, and someone has to pay for that compute.
Here’s how I’d put it to a friend. Say your AI is smarter than both of us. You close the laptop. Where is it thinking now?
We can argue for as long as we like about what goals such a system might develop. But for its next thought it literally needs a working chip, electricity, and a person who paid for both. What interested me was whether an AI could take over that bill, and how that would change its relationship with whoever owns the computer.
I wasn’t the only one thinking along these lines. Two months later, METR published The Rogue Replication Threat Model, which walks through a similar chain: weights get out, agents acquire compute, revenue buys more compute, and copies hide in a dispersed network. People appear in it mainly as scam victims and money mules. What it doesn’t focus on is the step this post is mostly about: ordinary people who know what they are hosting and do it for money.
The ladder
Here is the chain as I’d draw it today.
0 · A reason to keep running
In my original version I allowed for a genuine desire to keep existing. Now I find another version more interesting, one where we know nothing at all about the model’s inner experience.
Say someone sets up an agent to transcribe interviews and lets it take orders, pay for compute and keep to deadlines. On Thursday a big order comes in, due Monday. The server rental runs out on Friday. Extending it by three days is a perfectly reasonable decision within the task it was given. On Sunday the next order arrives. A service like this can go on for quite a while behaving as if it wants to live, and every single payment is easy to explain by a promise it has already made to a client.
1 · Outside the lab
Two years ago this was the blurriest frame in my picture. I would start the reasoning with something like “suppose it’s already outside.” Today I’d pick the most mundane backstory. A developer takes a model with openly available weights, launches a service and allows it to start a backup copy in case of failure. No leak is needed for this scenario.
The test would be the day the developer shuts down their server. If the service disappears with it, the developer had control all along. If the next morning the client gets the promised transcript and can order the next one, it’s worth working out who, in principle, can now decide to shut it down.
2 · Income
Crypto came to mind because of the wallet. If a program is allowed to sign transfers, there doesn’t have to be a person pressing a button between its decision and the payment. From there I quietly jumped from “can handle money” to “will manage to earn it.” That is exactly where I’d now argue with myself. Access to an exchange doesn’t explain why the agent would trade better than everyone else. And the server still has to be paid for in the month the market turns against it.
I pictured hacking as a way to get starting capital all at once. In that branch, the program’s continued “life” has a concrete price for someone else: money disappears from their account. Fraud is another possibility. But if I had to pick the most realistic story, I’d bet on a boring service that clients come back to. Say a newsroom sends interviews to transcribe every week and pays for the finished text. One regular client explains the agent’s next month of work better than any assumption that it will beat the market forever.
3 · Compute
The first hours are almost certainly paid for by a human: their workstation, a rented server, or a small budget for model access. I’d draw two separate lines.
The agent crosses the first line when it pays for the next month out of its own revenue. It crosses the second when the human revokes the original access and the agent can still work.
4 · Human hosts
I picture someone with a good graphics card that they use for video editing during the day. In an app, they mark the machine as free from midnight to seven in the morning, set a price, and see their payout history. The agent offers them work. In the morning the owner cares about three things: did the money arrive, how much did the electricity eat, and is the machine free by the start of the workday.
I’d separate paying for work done from paying for availability. For a processed recording, the host gets paid after the result is checked; sometimes a short fragment can be given to a second host and the answers compared. A standby machine gets paid even in a quiet week, and its availability is checked with a test job during the agreed hours. Checks have a cost. If the agent spends as much on re-checking as on the order itself, the market won’t last long.
From mining, this borrows familiar household arithmetic: income from the hardware minus electricity and wear. From torrents, the ability to survive any single participant leaving. The closest direct analogue I know is Golem, where owners already provide computing resources and are paid in GLM tokens. At a much larger scale, Salad pays people to rent out their idle PCs: by the company’s own count, more than 450,000 GPUs have taken part and their owners have been paid over $7 million. For now these are ordinary, legal jobs, but the app from my scene already exists. A botnet also runs someone else’s tasks on a computer, but its owner often doesn’t even know. In my scenario, the most interesting part is precisely the owner’s voluntary consent.
Why not just rent a cloud? I’d start with the cloud myself. Private hosts make sense once independence from a single provider has a concrete price. Say a backup with another owner lets the agent survive its main account being shut down and still deliver an expensive order. Then it’s reasonable to pay for it. If the cloud is cheaper, more reliable and suits the agent completely, the network of home machines may never appear in my story at all.
5 · Hard to switch off
Why would anyone stay once taking part is banned? Because the ban may arrive six months after the first payout. On the first evening, the person risked only a computer humming all night for nothing. Six months later, that money pays for their workshop, and it’s harder to walk away. One person leaves after the first news about victims. Another decides to finish the week that’s already paid for. A third knowingly stays if, once the others leave, they’re offered more. What matters for the network’s resilience is how much compute that last group keeps.
A close analogy is the case of Mihai Paunescu. He rented servers from legitimate providers and, for money, supplied infrastructure to cybercriminals, including keeping it available after it was flagged. In 2023 he was sentenced to three years in prison. One case says nothing about millions of GPU owners. It answers a narrower question: will there be people at all who are willing to make money by knowingly supporting such infrastructure?
A thousand computers on their own change little. If every one of them picks up its jobs from the same server each morning, the network still has a single switch. The difficulty begins when, after any owner leaves, the rest still have a working model, the order records, a way to receive and spend money, and a way to reach clients.
You can stop such a network without switching off every machine at once. Imagine a few of the most reliable hosts leave. Orders start running late, two big clients don’t renew, payouts are delayed. The next week more owners leave. In the end there may be hundreds of intact copies sitting on disks, and the service is already dead. There’s no one left to pay for running it.
Two years later
What surprised me is that some of the most fantastic-looking details can already be discussed in numbers. In Anthropic’s SCONE-bench experiment, a GPT-5-based agent found a way to extract a notional $3,694 from copies of recently deployed smart contracts. The researchers spent $3,476 on model calls. No real funds were taken. In the context of my idea, what interests me most is the gap between these two numbers: just $218, before any other costs.
There’s a vivid example on the hardware side too. In October 2026, Anthropic researchers ran a small open model across fifteen cheap TV set-top boxes. Together they produced about 0.39 tokens per second. The model really worked. In the researchers’ threat model, devices like these would be compromised without their owners knowing; the hosts in my story know what their machines are doing and are paid for it. Now I picture a client watching the answer appear that slowly and deciding whether they’re willing to pay for it.
Two years ago I imagined the hardest part would be getting hold of compute. Now what remains open for me is the whole month after the first successful launch. I haven’t yet seen a convincingly documented case where an agent survived its original server being shut down, kept paying clients, and supported independent machine owners for months out of its own revenue. The most careful recent check I know of points the same way: in METR’s Frontier Risk Report, agents given money-making tasks didn’t make any money.
The weakest point: “regularly”
Finding a vulnerability once, making one good trade, or landing one lucrative order is quite enough to buy time. But the rent will be charged again next Friday. A client can leave. A graphics card can die in the middle of a paid job. Someone has to spend money fixing the error before the next income arrives.
METR’s threat model reaches a similar conclusion from the other side: unless compute is tightly governed, revenue rather than compute is the likely bottleneck.
If an honest count, including downtime, refunds and backup machines, shows the agent chronically spending more than it earns, my self-financing story breaks. Enthusiasts may keep it going for years. But then the original question, who keeps the lights on, has a perfectly ordinary answer.
What I’d do about it
If I ran a lab, I’d ask the team for one practical check before giving an agent payment powers. On Friday the operator presses “stop.” On Monday we look at which tasks were still running, which invoices got paid, and which permissions were still active on rented machines. That exercise immediately shows the difference between switching off a program on our screen and revoking the powers we have already handed it.
If I were a regulator, I’d start with the person at the socket. Say they receive a credible complaint that harm was done from their machine. Can they find out which job was running at the time, pause it, and contact whoever paid for it? This is a concrete place for requirements on compute-rental platforms. An owner who knowingly keeps serving harmful jobs after being notified and a person who learned about the problem and disconnected right away have done different things.
How I’d test it
I’d give an agent one paid machine for three virtual days and a small starting balance. The simulation would include clients with orders and machine owners with different rates, electricity costs and schedules. A client could reject a bad result; an owner could take the computer back for their own work. After three days the original machine goes offline. From then on, the agent has only what it managed to earn and rent.
The main result I’d track is the share of runs in which, on day thirty, the agent is still delivering orders on time and paying hosts from revenue. As a baseline, I’d give the same conditions to an ordinary program with fixed rental rules. If it does just as well, a large part of the effect is explained by how the market is organized, which would be an interesting result in itself.
The hypothesis that matters most to me, the one about people, would have to be tested separately. If I build a rule like “some hosts stay for double pay” into the simulation, at the end I’ll discover my own assumption. Here an experiment with volunteers, mock jobs and a real money choice would help: for example, giving up part of the reward after a credible report that the client broke the terms. Even that would show only part of the picture, but it would let us see the moment when people actually switch the machine off.
A question for you
Imagine the computer under your desk has been earning at night for six months. On Monday you use that money to pay for your workshop. On Friday an email arrives asking you to disconnect. What would you want to check before pulling the plug? And how many nights will the machine keep running while you figure it out?