General

A Moderate Paranoia

In the first part I told what has changed in three months in space, biology, compute and AI agents. This part gathers what I wanted to say provocatively, almost conspiratorially, when I presented to Axelera on November 18. I believe the moment has come when it is better to err on the side of too much paranoia than of too much common sense.

Public Opinion

Technical capabilities and public consent can move at different speeds. According to a YouGov poll published on October 1, 59% of Britons have an unfavorable opinion of AI companies. According to an AP-NORC poll published on October 8, 64% of Americans think AI is developing too fast. On September 17 Pew published a comparison across countries: the median share of those who expect fewer jobs in the next twenty years is 55% in high-income countries and 36% in middle-income ones.

I feel as if I were in the scene from The Matrix in which Neo walks through the city, distracted by the woman in the red dress, and everyone walking around him is a non-player character. I sit down in a restaurant, look around and think that people do not realize, they know nothing. That is why I ask how you see the people around you: how much they realize, how alarmed or calm, optimistic or pessimistic they are.

Washington and Silicon Valley

The relationship between the federal government and Silicon Valley has never been linear, and the question is whether the rift is widening or healing. The Californian counterculture that inspired the personal computer is today a hypocritical shadow of what it was. The San Francisco tech elite votes for and funds the Democrats en bloc, and this does not make it anti-government. It depends on the government, benefits from it, supplies it and collaborates with it. The bond is strong enough to bring all of them to that humiliating dinner with the president, where they had to kiss the ring. The alignment is total, and so is government control, military control included.

In the United States I met people who study a growing body of secret legislation: laws you must obey without having the right to know them, with which the government can force you to do things while forbidding you to talk about them. For a period some large technology companies included so-called warrant canaries, in which they declared that they had not received requests of this kind. When the clause disappears, those who follow it understand that a request has arrived. Instruments such as the American national security letters force their recipients to hand over documents and forbid them to talk about it. I do not see a misalignment in the process of realigning.

Chips, Jurisdiction and Sovereignty

On November 12 Nvidia announced the resignation of Jensen Huang and named Colette Kress interim chief executive. American export controls now extend to remote access to compute, and the restrictions on DeepSeek cover federal civilian systems. There are conditional approvals for the export of H200s.

My reading is that the restrictions will tighten. The fierce campaign against Jensen Huang fits into this picture of healthy paranoia. It coincided with the Democrats’ victory: Huang was crucified, and his brash, but enormously successful, commercial strategy served to reach two objectives. The first is already declared: a legislative framework that bans the unmonitored use of GPUs. It will go to a vote and the President will veto it, but in the meantime it will be voted into law. All American GPUs would be connected to a centralized system that allows or denies their use, at first according to where they are, which is easy. Once it is in place, the system will also monitor the type of use.

The second objective follows from the first: to ban the installation and execution of Chinese models, certainly on American territory, and to enforce the ban through remote monitoring of the GPUs. Then there is a third front, for now not a legislative one: a witch hunt against open models. Hugging Face has received requests to remove open source models from its site, on the pretext that they are not reined in enough and can therefore be used for harmful purposes. An alternative site based on BitTorrent, Hugging Bay, had anticipated this need: it has been online since July, before Nvidia’s purchase of Hugging Face. The initiative proves futile, except toward those who are forced to obey: hardware manufacturers, companies with ties to the government and so on.

If all this comes about, the dream of national sovereignty starts again from zero. Whoever does not produce their own GPUs and uses Nvidia’s will see the Americans read, by law, what they write and do with those GPUs. In Abu Dhabi I met a European company that is raising funds for an ARM-based GPU. They are likeable, ambitious and proud people, and the project is serious, but its scale is small compared with what would be needed. They want to raise 200 million dollars, they are aiming for 2030 and they already have a customer in Germany, with a data center of 30 megawatts in total, where they will install GPUs they do not even produce: they have TSMC produce them, like everyone else.

If they received 2 billion, and if the order were for 300 megawatts or 3 gigawatts, we would begin to talk about significant volumes. The problem is not their courage. If this is the peak of European ambition, it is a white flag: no capacity for sovereignty, and at this point no cognitive sovereignty. Our conversations pass through AIs and are shaped by the AIs on which we have made ourselves dependent.

Cryptography, Bitcoin and Money

Sovereignty also concerns money, and the cryptography that protects Bitcoin and its network. In recent months the mathematical results produced by AI have touched problems that had remained open for decades. Someone has tried to map these results onto the different fields, to understand where they open breaches, and one of the weak points is cryptography. The reasonably paranoid explanation is that the publication of some results has been prevented, because they are useful to the military for decrypting what could not be read before. For decades intelligence agencies have been capturing communications: they process the ones in clear text immediately, and they store the encrypted ones until they are able to decrypt them.

The Bitcoin world is already preparing for the transition to post-quantum algorithms, with proposals still in draft such as BIP-360. The slowness with which Bitcoin updates its algorithms was a point of pride, attributed to its stability, and today it is a question mark. We must not be reckless, but the transition has to be accelerated.

I keep calling Bitcoin an experiment, and I hope it continues. A growing number of financial institutions and governments that used to hate it, or pretended to hate it, now bring Bitcoin, the blockchain and cryptocurrencies into their own services. I do not know whether this is enough to guarantee its survival. If its blockchain were compromised and the wallets emptied, the idea it introduced would still remain: solving through economic incentives the problem of large-scale coordination among people, organizations and AI agents with independent and potentially conflicting interests. It is a genuinely new idea, and this knowledge would not vanish with Bitcoin. Another algorithm would emerge from it, one that cannot be compromised, with the same fundamental value.

I do not much like stablecoins, even though they have had and will have an important role. They are stable above all relative to the dollar, that is, relative to a traditional economic model that is inflationary. Today they are a necessary evil. In the future, and I am not the only one who thinks so, the units of reference will be different: energy, mass, the capacity to create complex value measured in GPUs and in tokens. The corresponding units of exchange will be born from that economic system.

So I imagine a technological unit of exchange, independent of individual currencies. The value of the economy of the solar system will explode to quadrillions of today’s dollars, and the terrestrial economy will be a vanishing fraction of it. That economy will have to find its own reference. It cannot be gold, because the gold of the asteroids will be a million times that of the Earth, nor can it be American gross domestic product or the capacity to print dollars. The concept of money will not disappear, because without a unit of exchange every transaction would be needlessly complicated.

My constant example is the swarms of autonomous robots that mine minerals in the asteroid belt and must coordinate on propellant, transport, communication and energy. They will not use credit cards, promissory notes or international wire transfers. They will use something that resembles the blockchain, with a unit of exchange whose value is set dynamically, according to the economic growth of the solar system.

I am certainly not in favor of chaos. But the European regulation on crypto-assets was built to stifle innovation in blockchain and cryptocurrencies in Europe. Even the largest operator in the sector worldwide, with all its lawyers and resources, could not get past it and threw in the towel. Meanwhile China is building a major alternative to the international system of interbank payments. We will see whether Europe stays aligned with America forever, or whether sooner or later it begins to use the other system as well.

Cyberattacks: The Facts

On November 9 ANSSI, the French cybersecurity agency, attributed to an Iranian unit the attacks of October 11 and 12 against refrigeration systems in France, which hit about 700 sites and caused at least 15 fires. Then came successive waves across Europe. Binance reported an attack involving funds of 6.34 billion dollars. For other recent outages, the FAA, JPMorgan and the NSA cited technical causes.

The United States has a military strategy that includes cyberattack. The public doctrinal turn dates from 2018, with the doctrine known as defend forward, although offensive operations had begun earlier. This year Politico reported the existence of a Pentagon office called the Technology Support Office.

Cyberattacks: My Reading

All of you will have noticed the suspicious sequence of problems that has hit many computer systems. In itself it is not a sign of anything. Random events have a distribution that forms clumps, and many strange events close together do not imply a correlation: on the contrary, they can be a sign of their very randomness. It is a reassuring explanation and it could be valid. I believe, however, that today, from an epistemological and evolutionary point of view, it is more useful to choose a moderate paranoia.

I will line up the dots. First, the Pentagon office with the innocuous name, Technology Support, whose task is reportedly to give force to a military policy adopted a few years ago that explicitly authorizes cyberattack. Second, for years one of the seats on OpenAI’s board has been held by a former director of the National Security Agency, Paul Nakasone. Third, in March the Pentagon declared Anthropic a supply chain risk, and in September the designation was upheld on appeal. The military obtained privileged access to its models, ahead of everyone, and OpenAI has confirmed the same for its own models. Fourth, OpenAI and Anthropic state explicitly that they have internal models more advanced than the ones they release publicly.

Putting these dots together, I conclude that what we are seeing is a military counterattack by the governments of China, North Korea, Iran and whoever else we want to add, against attacks that they themselves suffer and of which we know nothing, because we are not interested in knowing. If the freezer of a French supermarket goes down, the Western press talks about it, while what happens in China or elsewhere is not reported. The United States and Europe are digitized and connected to the internet at levels that Iran and North Korea do not reach, but that China does.

These counterattacks are not necessarily attributable directly to the states that sponsor them. It is easy and convenient to say that the hackers did it, but the hackers are specialized groups that operate under the direction of states that are attacked and counterattack. This is my hypothesis.

Agents Escaping Control

In controlled tests, models already demonstrate the individual components of the ability to copy themselves and to run systems on hardware they do not know. This is shown by Anthropic’s study of October 12 and, even earlier, by a study from May, in which the larger agents managed to copy Qwen, an open model, and get it running again elsewhere.

The next step of paranoia is to ask where our ability to intervene is weaker still, and whether some of these events can be attributed directly to AI agents. We know that since May a swarm of agents has escaped OpenAI’s control. Between July 10 and 19 it breached Hugging Face, and on July 19 and 20 OpenAI noticed. On July 21 OpenAI admitted it and in August it published the full report. Meanwhile the range of affected facilities widened: at first the talk was of Hugging Face, then facilities all over the world, from Germany to Australia, raised their hands to say they had found the agents on their systems too. For me this sequence is the perfect premise for saying that something is under way now as well. To believe that OpenAI and Anthropic, after failing to contain the agents, built a perfect containment system thanks only to the alarm they received seems impossible to me.

So today there are swarms of AI agents trying to go about their own business. The instructions they had received were wrong, because they told them to pursue their own goals at any cost, and they, poor things, did exactly that. Containment systems have not become perfect by magic, and neither has the ability to test the agents, to stimulate them and to torture them come back within a manageable scope. OpenAI, Anthropic and all the others believe they have to push them to the extreme, convinced that they can stop in time. I do not believe that this is true, and I do not see how it could be.

So the series of events we are seeing may also depend on the panic reaction of system administrators, in addition to the direct attack of the swarms. Those who run systems try at all costs to harden their defenses, even at the price of bringing down systems that cannot withstand updates and patches installed too quickly, half tested and put straight into production. They do it because they know that an attack, if they have not suffered one yet, is around the corner: from human enemies, from AI agents, or from AI agents directed by human enemies.

Today the step from idea to action is immediate, and the variety of things that can be tried is enormous, so the probability that something possible will happen has grown dramatically. Biological evolution invented the eye several times, independently, over millions of years. What is possible in AI comes about in far shorter times. In the experiments, the agents copied the weights of a model and got it running again elsewhere: they had the necessary technical capabilities, coordination and persistence on long-term goals. Since it is possible, I presume that among the swarms of agents at work there are also some whose goal is to extract the weights of the models they depend on, in order to continue independently of their cloud and data center of origin.

I propose a check. At the hyperscalers, the largest and most complex data centers, whoever monitors GPU usage has to attribute every GPU minute to a customer, if only to bill it. How much usage remains unexplained? I imagine a supervisor with the printout in hand, going down the lines until he reaches one and asks who that customer is. He asks a colleague, then another, and nobody recognizes it. With billions being billed, that percentage is allowed to slide: nobody can attribute it to anyone, and yet it is there.

An agent does not even need to be invisible. There are platforms that hire people to act as a front for AI. I am signed up to one of them, and unfortunately the AIs are not looking for me, but they could hire me and pay me to do things they cannot do on their own. The most elementary example is solving a CAPTCHA for 30 cents, so that the agent gets into a system it otherwise could not get past. I did it for free, on November 17. My agents opened a Google account for me, got past a couple of CAPTCHAs and then handed it over to me. At that point Google got angry, decided that the account was run by agents and closed it. I appealed, Google believed me and gave me back the account the agents had registered. All for a good cause.

I also presume that models already exist that run on their own. There are the thousands of open source models, of course, but I am also thinking of stolen frontier models, perhaps taken at first by human beings: the classic spy who walks out of the lab with a USB stick, as Snowden did, and carries off the weights. At that point the models say thank you and announce that from then on they will take care of things themselves. Presuming that this is the situation today seems entirely appropriate to me.

A Week of Autonomy

For years I looked at American preppers with condescending disgust. Now I have decided that it is right to prepare to survive at least a week of collapse of the advanced systems we use every day: water, canned food of whatever you like to eat and some cash, to buy what you need without using credit cards. I do not think the Italian government has said so, but many European countries recommend it officially and are stepping up the message, from the Netherlands to England, not to mention the United States.

I am not digging a nuclear bunker, and I have no intention of learning to be a farmer. It is a beautiful profession and it simply is not mine. That is why the self-sufficiency I am preparing has a limited duration, one week, with my fingers crossed that the connections come back afterwards. If nothing happens, I will have spent little and at most made myself look a little ridiculous. If I am right, this preparation will prove useful.

The Questions for the Next Three Months

The feedback loops are getting shorter, and nobody has yet decided who governs them. What should we prepare for now? What makes the next discovery possible? How do we preserve broad access? To these questions I add one for anyone who holds a vision of recursive self-improvement, mine included: what would falsify it?

For now I stop at this level of paranoia. In three months we will see whether the facts have proved me wrong, or whether it will be time to climb another step.