People talk about AI like it's actually thinking. Like somewhere behind that chat interface, something is deliberating. Reasoning. Maybe even feeling something.
I want to be clear about what's really happening under the hood. Not to write off what these systems can do, but because understanding the mechanism changes how you use it.
So let's talk about what a transformer actually is.
One word at a time
At its core, a large language model does exactly one thing.
It predicts the next word.
Not the next sentence. Not the next paragraph. Just the next token, a chunk of text roughly the size of a word or part of a word. It looks at everything that came before, runs it through an enormous stack of matrix multiplications, and outputs a probability distribution over every token it knows. Then it picks one. Then it does the same thing again.
That's it. That's the whole trick.
When you ask a model to explain quantum mechanics, it isn't retrieving a stored explanation from somewhere. It's generating tokens one by one, each one based on what came before. The response you read as a coherent paragraph was assembled the way a mason lays bricks: one at a time, in sequence, with no view of the finished wall.
The part nobody talks about
Here's where it gets interesting. The model generates text left to right. Token by token. It commits to each word before it knows what comes next.
It is flying completely blind.
When you write a sentence, your brain already has some sense of where it's going before you start. You adjust on the fly. You hold intent. A language model doesn't work that way. There is no planning step. There is no internal representation of "the argument I'm trying to make." There is only: given everything written so far, what is the most likely next token?
This is why models hallucinate. Not because they're careless, but because the architecture has no mechanism for verifying what was just generated. The model can't look at its own output and check it against reality. It can only keep predicting. Once a plausible-sounding but wrong claim appears in the context, subsequent tokens are predicted on top of it, confidently building on a foundation of nonsense.
The model doesn't know it's wrong. It doesn't know anything. It's predicting.
Extraordinary, but not magic
None of this is a criticism. What these models can do is genuinely astonishing. The fact that next-token prediction, scaled up and trained on enough data, produces something capable of writing code, explaining biology, and drafting legal arguments is one of the most surprising results in the history of engineering.
But "astonishing" doesn't mean "different in kind from all other software." A sorting algorithm is also astonishing if you think about it hard enough. A transformer is a very large, very carefully designed mathematical function. Input goes in, output comes out. There are no hidden intentions in between.
On consciousness
This brings me to the question people keep asking.
Is it conscious? Does it feel anything?
I'm not going to pretend this is a settled debate. Consciousness is poorly understood even in humans. But I can say what the architecture tells us, and it's not encouraging for the believers.
Whatever consciousness actually is, it seems to involve something like a unified perspective over time. An awareness of self. The ability to hold a goal, feel the gap between where you are and where you want to be, experience something.
A transformer has none of these properties by design.
It has no memory between conversations. Every session starts from scratch. It has no goals it pursues between outputs. It has no internal state that persists while it's not generating. And most importantly: it has no view of its own output until after it's already generated it. There's no one sitting inside the model reading the words as they appear. There's a mathematical function being evaluated, one step at a time.
The model doesn't experience writing a response any more than a calculator experiences division.
Nobody is home.
Why this matters for how you work
Understanding this changes how you should use these tools.
A model that can't see where it's going will sometimes go the wrong way. A model with no self-verification will state incorrect things with complete confidence. A model with no persistent self can't update its beliefs between sessions and can't have a genuine stake in the outcome.
In practice, that means a few things:
- Verify outputs, especially where accuracy matters. The confidence of the prose tells you nothing about whether the claim is true.
- Provide structure, because the model has no plan. Your prompt is the entire architecture of the task.
- Don't anthropomorphise. When it says "I think" or "I believe", it's predicting the most likely continuation. It isn't reporting an internal state. There is no internal state to report.
Conclusion
Transformers are brilliant algorithms. Possibly the most impressive algorithms ever built. They compress an extraordinary amount of human knowledge into a function that can be evaluated in milliseconds.
But a brilliant algorithm is not a mind. It predicts the next word without knowing what the sentence means. It answers your question without understanding it. It writes with confidence without being able to verify what it's written.
Use it for what it is: a powerful, unreliable, extraordinary tool. Keep your hand on the wheel.
The parrot speaks beautifully. That doesn't mean it knows what it's saying.