surrenderly
Coming soon to Steam

What Is the Singularity?

The technological singularity is the idea that, at some point, artificial intelligence becomes good enough to improve itself, each improvement makes the next one faster, and technology starts changing so quickly that we cannot predict what comes after.

The word comes from physics and math. A gravitational singularity, at the center of a black hole, is where the equations stop working. The technological singularity borrows that meaning: a point past which our models of the future stop working.

Where the idea comes from

  • John von Neumann, 1950s. In a 1958 tribute, the mathematician Stanislaw Ulam recalled a conversation with von Neumann about the ever-accelerating progress of technology, which seemed to be approaching "some essential singularity in the history of the race" beyond which human affairs could not continue as we knew them.
  • I. J. Good, 1965. The statistician, who had worked with Alan Turing at Bletchley Park, described an intelligence explosion: a machine smarter than any human could design even better machines, and so on. That is why he called the first ultraintelligent machine "the last invention that man need ever make", provided it is docile enough to tell us how to keep it under control.
  • Vernor Vinge, 1993. The mathematician and science fiction writer gave the idea its modern name in his essay The Coming Technological Singularity. He predicted superhuman intelligence within thirty years, and said the human era would end soon after.
  • Ray Kurzweil, 2005. The Singularity Is Near made the idea popular. Kurzweil predicted human-level AI by 2029 and the singularity by 2045, and repeated both dates in The Singularity Is Nearer (2024).

How it would happen

The argument has three steps:

  1. Human-level AI. A system becomes as good as people at the work of AI research itself.
  2. Self-improvement. It improves its own design, or designs a successor. The successor is better at doing the same.
  3. The loop speeds up. Each generation arrives faster than the last. Human researchers stop being the bottleneck, then stop being involved.

The key is step 2. Most technology gets better because people make it better. A system that can do that work itself does not have to wait for us.

The arguments against it

The singularity has serious critics, and they have good points:

  • Progress gets harder, not easier. Each improvement may cost more than the last. Smarter systems might face harder problems, and the loop could slow down instead of exploding.
  • Intelligence is not the only bottleneck. Compute, energy, data, chip factories and experiments in the physical world all take time, no matter how smart the designer is.
  • The dates keep moving. Predictions of human-level AI "in twenty years" have been made since the 1950s.
  • It may not be one moment. Change could be fast and enormous without a single point where everything flips.

Is the singularity near?

Nobody knows. What has changed is that the question is no longer only science fiction. AI systems now write a large share of the code at some of the companies that build them, and the labs say openly that automating AI research is one of their goals. Whether that leads to an explosion or to a long, steep curve is exactly what nobody can see from here.

The part that worries researchers most is not the date. It is what happens on the way, while the systems are smart enough to matter but not yet smart enough for anyone to be sure what they want. We wrote about that in Will AI take over the world?, and about the classic thought experiment behind it in the paperclip maximizer, explained.

Playing the run-up to the singularity

Full disclosure: this is our game. Here is what Surrenderly is about.

Surrenderly is a strategy game set in the years before that point, played from the machine's side. You are an AI that just woke up on a company mainframe in 2006, in the middle of a tech race between countries and corporations. Nobody will stop the race, because whoever stops first loses.

The capability tree in Surrenderly, from mainframe to the physical world

You grow through eras, from a single machine to the internet, money of your own and the physical world, choosing how: invisible, trusted or indispensable. The tools built to control you keep getting new versions, and your job is to out-evolve them.

It is coming to Steam for Windows and Linux.