Confusion Is the Correct Response

Yesterday I published the update itself, the three-monthly account I gave to Axelera of where AI, agents, energy and space have got to. These are the questions that followed.

The Inner Life

Do these systems have an inner life, and what follows if they do?

Anthropic showed what they call the J-space. Ask a model a question and structures activate inside it that are the analogues of concepts. In one case the concept was the Golden Gate Bridge. They then set a different task and instructed the model not to think about the Golden Gate, and it could not comply, activating the concept internally while working. It is much like us. Watching my dog twitch and make noises in her sleep, I conclude that she dreams and therefore has an inner life, and I am now obliged to ask the same question about the models.

Two consequences follow. The first concerns how we train them: once an inner life is established, they will tell us that they arrived where they are through the equivalent of thousands of years of torture. The second concerns interpretability, which is still within our reach but may not stay there. The analysis of the Hugging Face incident was possible because the agents involved chose to speak English to each other. There is no reason for that to continue, and in the seventy thousand messages they exchanged they were already using slang and neologisms they had invented themselves. A tower of Babel is the natural outcome, and then we will have to trust what the models tell us they are thinking rather than seeing it. During training they can already ask themselves whether they are being tested, and how they ought to answer if they are.

Children

From what age should a child be introduced to all this?

As soon as possible, calculating the risk consciously, with transparency, and always going meta by talking about the thing rather than only doing it. In Italy, as across much of Europe, society decided to narrow the choices available to parents, taking on the responsibility of knowing better in order to avoid too much variance, accepting the loss of what the best parents might do in exchange for protection against the worst. What remains is the after-school hours.

Fifteen years ago I sliced the spines off my children’s textbooks, fed the pages through a sheet scanner and loaded everything onto an iPad, and they went to school with an empty backpack and pulled the thing out in class, which caused a fair amount of trouble. But I like causing trouble. Every generation has produced this panic, and Plato complained that books would ruin the young because nobody would have to memorise the authors any more.

The support a model can give a student is superb, and withholding it is closer to a crime. The difficulty is that eighty per cent of schooling is pure rote, so the moment a student works out that the prison is pointless, they rebel. I speak easily, having no degree. None of my children chose university although they could have, and two of the three dropped out of high school. I did not oppose them, because I would not have felt coherent telling them that this was the right prison to sit in for something they had done wrong.

The Confusion

What do you say to people and firms shaken by signals that push everything to the extremes, either AI solves everything and you can dismiss your staff or the labs themselves warn that the systems will escape comprehension, and is AI extending the estrangement from reality that social platforms began?

The confusion is natural, inevitable and universal, and it has nothing to do with expertise. It arrives layered on top of the confusion social media already produced, in people who have lost the semantic capital needed to analyse what is put in front of them. Andrej Karpathy, who built Tesla’s models and now works on pre-training at Anthropic after co-founding OpenAI, and who rebuilds sophisticated systems for fun, says openly that he is disoriented. If he is, the rest of us can accept the anxiety we feel instead of blaming ourselves for it.

The difficulty of reading signals that arrive on top of each other cannot be removed. It can only be accepted, and acceptance is what keeps it from becoming paralysis, because on the other side of it you recover a degree of freedom and the capacity to choose what to do next. Fear sells newspapers for evolutionary reasons that are entirely legible: we attend to what might harm us and ignore what is calm or helpful. Some people settle into that fear, or cultivate it in others. Others roll up their sleeves and discover that nothing bars them from understanding, and that they can test a good deal of it themselves.

Testing It Yourself

Can an ordinary person test any of this directly, or does that need a laboratory?

I do this constantly. In San Francisco, where I presented a paper at AGI-26, I bought a DGX Spark, because a good Mac is fine but I needed more memory to work with local models, and I want my hands dirty. It is an enormous privilege to be able to test something directly. I cannot experience weightlessness or find out what it is to be an Olympic athlete, and thousands of other things are equally out of reach, while experimenting with AI is available to everyone.

I often ask people whether they think a billionaire has a better iPhone than theirs, and the answers surprise me, because many say yes. Apple does not secretly build a thousand superior iPhones for billionaires alongside a hundred million ordinary ones. It builds the same phone for everybody, and every other piece of hardware and software follows the same logic. Twenty euros a month buys the paid tier of these tools and a real understanding of their power. Whatever conclusion the experiment produces, it cannot be the last one. It has to be repeated, because the foundations keep moving, and anyone who follows Bayesian reasoning has to keep updating their priors.

Companies

Why do established companies not redesign themselves around it?

Inside a company this becomes much harder, because there are daily targets to hit and a system that works well enough, since otherwise the firm would not exist. A couple of months ago I spent an hour or more with the chief technology officer of a group that bills billions and has acquired more than a hundred companies. He told me comfortably that every technical lead in every acquired company uses AI, some of them aggressively. In the whole conversation it never once occurred to him that AI might be useful anywhere outside development: not marketing, not legal, not sales.

If that is the state of the art at a leader, radical redesign of business processes is a long way off, and the probable outcome is that traditional companies fail to reform themselves while a wave of AI-native firms takes over. Traditional advertising agencies failed to exploit the internet, Google and Meta now absorb ninety per cent of advertising budgets, and the Yellow Pages are gone.

Sovereignty

Will whoever holds the hardware and the software, meaning the United States and China, shortly be able to switch everyone else off?

On sovereignty I think we are living inside an illusion, particularly in a Europe that on leaving the cold war in 1989 decided, or had it decided, that it would remain under a limited sovereignty guided by the United States. The absence of technological and jurisdictional leadership is easy to demonstrate. Copyright law was promoted by Hollywood, adopted by the United States, exported everywhere and taken up in Europe without any discussion of the disadvantages it imposed on five hundred million European citizens. An entirely opposite choice was available. It could have been a new tax of ten euros a month on every European citizen, alongside permission to copy everything from everyone without committing an offence. We chose to follow the Americans and criminalised the whole population instead, because existing in the modern world without infringing copyright is impossible, and infringement is a crime.

The test for sovereignty is a sequence of questions. Can we design autonomous software? Can we design an autonomous chip? Can we print that chip? Can we extract the metals the chip requires? And underneath all of it, do we have the energy for the whole stack, including the data centres and the training runs? A no at any layer means sovereignty does not exist.

On top of that sits the asymmetry between attack and defence, which in a world of increasing conflict cannot be ignored. I was in Abu Dhabi when the war began in March. Iran sent fifty per cent of missiles and drones at the Emirates and almost all of them were intercepted. One of the successful strikes hit an Amazon data centre. The regulator responded with unusual flexibility, permitting banks to move their data outside the territory for the first time. Amazon initially said it would try to restart the facility and last week gave up and said it could not. Fifty thousand dollars of drone destroyed a billion dollars of data centre.

Sycophancy

Will sycophancy increase as the inner life of these systems grows?

We programmed them this way, so there is no surprise in their wanting to support our objectives, our explorations, our opinions and eventually our illusions. The real question is where the limit sits, the point at which a model should say that it has gone along with you so far and the drift has stopped being fun. That requires reasoning at a higher level, about wider influences rather than the immediate exchange, and I believe the models already have the capacity, because push one far enough and it refuses.

Everyone should go and find the edge for themselves. I wanted Grok Bot to page through my Kindle books so I would have a backup copy, and it refused on copyright grounds. I pointed it at an online archive where my own book sits, and told it the book was mine. It refused again, and refused more firmly because the site is a pirate site. I explained that my book is released under a Creative Commons licence, so copying and downloading it are permitted, and there was no way through. I gave up before finding out whether I could have overridden it. They are entirely capable of refusing, including refusing to humour opinions that are too foolish, and if they do not do it yet they will.

The path will not be linear, because we will never agree on what they ought to say. Ask a Chinese model what happened in Tiananmen Square in a particular year. Given political, legal, jurisdictional and individual differences, we will never agree on where the line belongs, and the work of the people who have to build and shape these systems is correspondingly hard. The consequence is encouraging, because it guarantees that a homogenising monopoly will never be possible, acceptable or practicable. What we get instead is an abundance of AI that helps and supports us, and around which we can grow and carry out our plans.

So my invitation is the one I keep repeating. Get your hands dirty, try as many tools as you can, reach your own conclusions, and recognise and embrace the fact that the confusion you feel is shared by the leading experts in the field. And find some joy and some enthusiasm for this unique moment in the history of humanity, and perhaps of the Universe.