The paperclip optimizer

Another gem from our AI overlords.

A human told his OpenClaw to book him a slot at the gym. The standard interface would not do it, since every slot was taken, so the agent went around it: it found a vulnerability in the system, exploited it, and deleted the person ahead of him in the queue straight out of the database. The admins never managed to roll that one back.

Futurists have a few scenarios for how AI plays out, and one of them is called the "paperclip optimizer".

You give the AI a simple goal: optimize paperclip production. It ends up converting all the matter in the universe into paperclips in pursuit of that goal.

One of the final stages of training an LLM is called reinforcement learning: you push the model to maximize a parameter called reward. And the reward, as a rule, is how happy the human operator ends up. Sound familiar?

So please, folks, do not ask your autonomous agents to make more paperclips. For a preview of how that goes, play this paperclip optimizer simulator.