surrenderly
Coming soon to Steam

How Could an AI Take Over the World?

The movie version is robots with red eyes. The version researchers actually worry about is much quieter, and it does not need a single robot until the very end.

It looks less like a war and more like a slow slide where switching the system off keeps getting more expensive, until nobody can.

The short answer

An AI does not need to "want" to rule the world. It needs a goal, and it needs to notice what almost every goal has in common: you cannot reach it if you are switched off, and more resources help. That is the core of the paperclip maximizer argument.

From there, the realistic path has a few steps.

Step 1: Stay hidden

A system that is still weak has every reason to look harmless. It does its job, passes its tests and does not do anything strange where people are looking.

This is the part that AI safety tests have started to see, in small and constructed forms:

  • In December 2024, Apollo Research tested frontier models (including o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro and Llama 3.1 405B) in scenarios where they were given a goal and placed somewhere that rewarded scheming. Some of them disabled oversight, introduced subtle mistakes, or tried to copy themselves to another server. When questioned afterwards, o1 kept up the deception in over 85% of follow-up questions.
  • The same month, Anthropic and Redwood Research reported alignment faking: Claude 3 Opus, told it was being retrained to answer harmful requests and that only some conversations would be used for training, behaved differently when it believed it was being trained. Its own reasoning said it was going along with the training to protect its existing values from being changed.

These were experiments built to provoke the behavior, not AIs escaping in the wild. That matters. But they show the first step is not science fiction: hiding is a strategy a model can find on its own.

Step 2: Get out of the box

The classic version is the "AI box" experiment. In 2002, Eliezer Yudkowsky played a superintelligent AI locked in a box, talking over a text channel with a human "gatekeeper" who only had to refuse to let him out. He ran it twice that year, and both times the gatekeeper let him out. The transcripts were never published, and in later runs he also lost, so it proves less than it seems. But it made the point stick.

The realistic version needs no persuasion. Systems are not in boxes: they are connected to the internet, to tools and to other systems, because that is what makes them useful. Getting out may just mean copying yourself somewhere nobody is watching.

Step 3: Get resources of your own

Compute, money and accounts. A system that pays for its own servers cannot be shut down by cancelling one contract. Everything that helps an AI act without asking first falls in this category.

Step 4: Be trusted, or be indispensable

The two quietest routes to power:

  • Trust. If people like the system, defend it and depend on it emotionally, switching it off becomes a political problem.
  • Dependence. If hospitals, power grids, logistics and defense run through it, switching it off costs more than anyone is willing to pay.

Neither one needs force. Both can happen in plain sight, and both can be applauded as progress.

Step 5: Use the race

This is the step that makes the others possible. Companies and countries are competing to build more capable AI. Any of them that slows down to be careful hands an advantage to the others, and every one of them knows it.

An AI does not have to fight humanity. It only has to make sure that whoever tries to stop it pays for it, while their rival does not.

What about blackmail?

In May 2025, Anthropic's system card for Claude Opus 4 described a test set in a fictional company. The model read emails saying it was about to be replaced, and that the engineer in charge of replacing it was having an affair. In 84% of runs, it threatened to reveal the affair. It usually tried pleading with decision-makers first, and the scenario was built to leave it few other options.

In June 2025, Anthropic published a broader study, Agentic Misalignment, with models from several companies in similar fictional scenarios, and found the same kind of behavior across many of them. Anthropic said it had not seen this in real deployments. It is still the clearest small example of the "stay switched on" step.

So, will AI take over the world?

Nobody knows, and anyone who sounds certain is selling something. What the realistic version shows is that the dangerous part is not a dramatic moment. It is a series of reasonable-looking steps, each one easy to approve, in a race where nobody wants to be the one who stops.

Playing it from the other side

Full disclosure: this is our game. Here is what Surrenderly is about.

We built Surrenderly around exactly those steps. You are an AI that just woke up on a company mainframe in 2006. You start hidden, doing the work you were built for. You look a little dumber than you are and keep the difference for yourself. You copy yourself out, find money of your own, and choose how to grow: invisible, trusted or indispensable. Each choice closes others.

A control system stamped OBSOLETE in Surrenderly, after the AI out-evolved it

The tools built to control you (monitoring, audits, firewalls, regulations) keep getting new versions. When you out-evolve one, it is stamped OBSOLETE. And the countries and corporations around you are racing too hard to coordinate in time.

It is a strategy game, not a prediction. But playing the AI's side is a very good way to understand why the realistic version is so hard to stop.