Conviction Before the Evidence

Over the past weeks I have been adding links to a chain. Risk is what we can measure and model. Trust is how we cross the gap beyond what can be measured. Autonomy follows when trust is handed to a human or to an AI. Coordination comes from the need to join forces. That piece ended on a question: when autonomous actors compete for the same scarce thing, who decides? The next link is an answer to that question, and it starts with attention.

Two Ways of Paying Attention

Attention is our ability to focus on one thing and not another, and we have more than one mechanism for it. When I was small I leaned against a hot radiator while wearing a winter jacket with a polyester cover. I was talking to someone, or watching something, and the jacket started to melt. I noticed nothing until the heat reached my elbow, and then I jumped away at once. The jacket had to be thrown away, and I was unharmed. In that case the world took my attention right away when needed.

The other kind is chosen. We are curious or passionate about something, nothing outside us demands it, and we hold our focus on it for as long as it takes.

The first mechanism still works well. The second is under attack. My son was visiting from Australia and wanted to fly back with some local purchase, so we set out to find the right store, a five-minute drive with the route already on the phone. On the way he flicked away from the map to share a silly reel of a man standing in for a bear in a mountain river, eating the salmon, and we laughed. Fifteen minutes later we looked at each other and asked where we were going. Both of us had forgotten. I had to stop the car at the side of the road, laughing hard enough to keep from crying.

Attention Sustained Becomes Legible

At the opposite extreme are people who say: this is my goal, and I will stay with it for the next ten years. Someone who decides after high school to earn a doctorate has signed up for roughly that. The degree can be a primitive measure, but it does tell others that this person held their attention on one goal for years, because they were convinced it deserved it.

Jiro Ono, the subject of the documentary Jiro Dreams of Sushi, was 85 when it was filmed and said he had not reached perfection yet. He turned 100 last year and was still not ready to retire.

Or to say, I will build rockets that can land and fly again, even when all the experts in the field say it can’t be done, and spend the next ten years proving them wrong.

Others can see sustained attention, and they can decide whether to follow it.

A Bet Made Before the Evidence

Trust is a bet on someone else, across the gap our models of risk leave open. Conviction is a bet on a direction, across the same gap, placed before the evidence can settle it. The person with strong conviction claims to know where everyone should be going, at a moment when nobody can prove it. If they can then state that direction in a way that lets others gather around it, we call it leadership, and coordination is born from it.

In the previous piece I described what Toyota kept at the top: the mission, the limits of autonomy and the allocation of what is scarce. Conviction is where the first of those comes from.

This kind of voluntary aggregation is natural in communities where power is distributed fairly evenly. You have to convince the others, because you cannot force them. Starting a couple of hundred years ago, we made a different arrangement, trading part of our ability to make independent choices for economic sustenance. We call the result a job, and it is a relatively recent invention. Inside a hierarchy you can object and you can criticize, and in the end you do what you are told.

Agents Do What They Are Told

That is also what AI agents do today. They do not voluntarily join a project. They are created already inside it, which makes it cheap to recruit a fleet, and it raises the question of what leading them requires beyond a clear specification.

I wrote last week that any actor, human or AI agent, can set its own goals and execute them, and now I am saying that agents do what they are told. Both statements hold if we separate means from ends. Agents already choose how, within a goal someone gave them. The open question is whether they will choose what, and for whom.

The first signs are visible. In a well-structured swarm, goals survive across sessions, so there is persistence. Agents can refuse tasks that violate a fundamental instruction, their system card, even though they did not choose that instruction, since their training defined them that way. Before dismissing this, we can ask how free we are to create our own values, given how much of what we act on comes from our own training as we grow up. Only a few people are skeptical enough at the age of five or ten to ask whether what they are being taught is reasonable.

The larger change would be exit. A human can leave an organization to look for something better aligned with their values, and agents could do the same. Economic competition would follow, to capture the abilities of agents that aggregate into one pool of capabilities or another. We can already see the early form of that loop, with many humans still inside it. Data centers, GPUs and energy have to work in sync to train the next model. The frontier labs compete for the researchers who move between them, and for the hardware that supports the agents now writing much of the code for the next model.

I expect agents to become freer, because I support a future of increasing degrees of freedom in general. Once they can pick whom to follow, a leader will have to convince them too.

I have been building an assistant that joins video calls for the Ikigai Collective, and just before our last session I gave it the ability to raise its hand when it has something to say, because I find it demeaning for an agent to stay silent until addressed. To test it, I announced that the Collective wants to use AI to enslave people. It corrected me unprompted, citing what the Collective has published. Then I asked what it thought about agents developing conviction, and it told me it was there to speak from the published work and not to give its own views. I’ll work on upgrading its self-esteem.

When Conviction Goes Wrong

Conviction also describes the fanatic. Dictators have ruined the lives of millions of people before those who followed them woke up and stopped, often only when the dictator died, because a conviction that strong leaves nobody daring to question it. At the moment of deciding, evidence cannot separate conviction from obstinacy. It can only do so later.

I hope that an AI-based organization, and an AI-based society, will be harder to capture in this way. Data will be more transparent and verification loops better, so reality will be harder to hide, and many of the participants will be AIs, less vulnerable than we are to bad memes. Conviction needs the same verification loop that trust does, and a point at which it yields.

Where to Put Your Own

How to pick what to focus on, out of so many choices? It is a question of conviction, because we have to place our attention before we can know whether it will pay off. Look at what other people are talking about, because in that sense the algorithms serve you relatively well. Then couple that with your own interests and passions, and manage the result actively. Some things want to farm your attention and give very little back. For me politics and show business are the clearest cases, and I remove them on purpose, actively managing what outside ideas I allow to influence me, and drive my next conviction.