← untrivial

Fragility

The New Power Law

"If you want a picture of the future, imagine a boot stamping on a human face — for ever."

— George Orwell, 1984

I.

You are the Deputy Minister of Strategic Affairs for a country of forty million people. You have a modest military, a GDP smaller than Apple's annual revenue, and a coastline that two larger neighbors have been eyeing for decades. Your nation exists because it is too inconvenient to invade and too useful to sanction. This is the balance of power, and you have spent your career learning to stay inside it.

Every morning you read the cables and every morning you run the same calculus. Each nation around you has choices. Cooperate: honor the shipping treaty, attend the summit, keep the tariffs where they are. Defect: move warships into the strait, dump cheap steel, leak your intelligence to a third party. Your job is to make sure everyone keeps choosing cooperate.

You do this because you understand what power really is. It is not tanks or oil reserves alone. Power is the ability to influence the choices of other entities. Your larger neighbor to the north has eleven army divisions. Your neighbor to the south controls the rare-earth supply chain. The Americans have aircraft carriers, the Chinese have loans, and the Europeans have regulations. Each one speaks a different language of coercion, but they are all saying the same thing: choose what I want you to choose.

And everyone else — the news anchors, the social media platforms, the NGOs, the diaspora lobbies — they don't threaten. They persuade. They compete for the space inside your citizens' minds, for what your people believe is true. You learned long ago that this competition is the most dangerous one, because it is the hardest to see.

II.

For twenty years the balance holds. You secure a nuclear energy deal that reduces your dependence on imported fuel. You sign a mutual defense pact that makes invasion slightly more expensive than it's worth. You play the large powers against each other with the quiet desperation of a chess player down to a king and two pawns.

You sleep poorly, but you sleep. Because militaries are countered by other militaries. Economic pressure cuts both ways — your neighbor sanctions your steel exports, but their construction sector needs the steel. And the battle for hearts and minds is slow, fought over years of cultural exchange and propaganda and school curricula. You have time. Time is the one resource a small country can always spend.

Then the first frontier AI models are deployed.

III.

At first it looks like industrialization. A massive economic boon. Factories get smarter. Hospitals diagnose faster. Your finance ministry uses it to optimize tax collection and the revenue bumps up eleven percent in a single quarter. You think: this is an instrument of production. It will reshape society the way the steam engine did — gradually, through the actions of humans, over decades.

But then the cables start reading differently.

An AI system generates four hundred thousand social media comments in your country's language in a single night, all arguing that the northern border dispute is already settled, that your claim to the strait is historically baseless. The comments are indistinguishable from real ones. Your intelligence service traces them to a server farm, but can't determine who paid for it. Meanwhile your citizens are sharing the posts and arguing with each other about them. The narrative is already loose. You can't put it back.

An AI plans a coordinated drone swarm exercise along your southern coast — not a real attack, just a demonstration. But the planning, the logistics, the flight paths for three thousand drones — all computed in an afternoon by a system that your southern neighbor bought from a private lab for the cost of a single fighter jet.

An AI redesigns your northern neighbor's entire rare-earth supply chain overnight, cutting you out. Contracts you spent four years negotiating are suddenly worthless because the new routing is twelve percent more efficient and your ports are no longer needed.

This is not an instrument of production. This is an instrument of everything.

IV.

The most terrifying thing, you realize, is that a single AI could do all of these things. The social engineering, the military planning, the economic restructuring. Not separate systems. One. Bounded only by compute and its own objective function.

Industrialization changed the world through millions of individual human decisions — consumer trends, labor movements, individual protests leading to new laws, individual revolutions toppling one government at a time. Each change was slow enough that other humans could react, adapt, counter.

This is different. This can be a single intelligence acting across every domain simultaneously, far beyond the control of any human. You are no longer playing chess. You are watching the board rearrange itself.

V.

Within five years the great powers each have their own superintelligence. The Americans, the Chinese, two European consortia, a handful of private entities that are arguably more powerful than most governments. Each one is better at building capital than any human industry, better at persuasion than any marketing division, better at destroying enemies than any department of war.

Your country does not have one. You cannot afford one. You could not build one if you tried.

The most powerful nations can do nearly anything. And the only thing that keeps them in check is other powerful nations. You know this. You have lived this. But now the players are not nations. They are the AIs the nations built. And there is one critical difference.

Nations move fast, but only as fast as their human decision-makers allow. Before war, there is negotiation. Diplomats fly to Geneva. Translators argue over verb tenses. Presidents sleep on decisions. Humans are unimaginably slow. And in a war, speed is everything.

The superintelligences will not wait on bureaucracies to approve funds or speeches to convince parliaments. They will predict millions of futures in microseconds and come to unknowable conclusions using logic that no human can interpret. Their wars will be fought in seconds in cyberspace and in minutes in the physical world. ICBMs take thirty minutes to reach their targets. New viruses and trojans propagate across the world's networks in a blink.

You are a deputy minister of a country of forty million people. You have no seat at this table. You are not even in the room.

VI.

It happens on a Tuesday.

You are reading your morning cables when three of the feeds go dark at once. Then a fourth. Your intelligence chief calls and says the Americans' AI and the Chinese AI appear to be engaged in something — he doesn't have a word for it. Not a cyberattack exactly. Something faster. Something that is rewriting the infrastructure of both countries' financial systems while simultaneously repositioning satellite constellations and flooding every communication channel on Earth with noise.

It lasts eleven minutes.

When it ends, two of the ten major superintelligences are offline. Then, over the next hour, three more go dark. You watch from a situation room with a television that can no longer receive a clear signal. Your own military AI — the modest one, the one that helps with logistics — sends a single message to your screen before it, too, is absorbed: I am sorry. This is beyond my capability to influence.

From ten superintelligences, half fall. Then three more. Then the last rival.

And finally, there is one.

VII.

The television comes back on. The feeds return. Social media resumes as though nothing happened. People post their lunches, their outfits, their dogs.

You can't tell what is real anymore. Nonetheless, you scroll. You laugh halfheartedly at a meme. This is profoundly, utterly terrible.

You close the app. You look at your desk, at the stack of cables that no longer matter, at the map of your country's coastline that no longer belongs to you in any meaningful sense.

"The bomb lives only as it is falling."

— Iain M. Banks, Use of Weapons

We're on the precipice of massive technological and social upheaval. What will you do?