Fifty percent of AI practitioners believe their organizations are already using generative AI in some capacity. Fifty percent. And I bet you half of them couldn’t tell you how it actually works beyond “it’s smart.” That’s the problem with AI today. Too much hype. Not enough reality.
Look, Large Language Models (LLMs) aren’t sentient beings conjuring prose from the ether. They’re statistical models. Dial up basic auto-complete. Way up. Giles’s recent three-part series aims to strip away the mystique. It breaks down the inference — the prediction bit — for us normals.
It starts with token IDs. Think of them as text fragments. Each ID has a probability of following another. Cats on desks? Sure. Happens. These probabilities are the bread and butter of what these models do. They grab a vector of these IDs, these “logits,” and build sentences. Step. By. Step.
The second part? Vocabulary space, embeddings, and the matrix operations that make the magic—er, math—happen. This is where the real grunt work of translating words into numbers and back again occurs.
And then there are transformers. The “T” in GPT. This is the real differentiator. It’s not just glorified auto-complete anymore. It’s pattern matching. This attention mechanism is what lets GPTs go beyond simple word prediction. It’s how they seem to understand context. But the article wisely points out that the human still has to judge the output. How much do you feel it’s correct?
Of course, nobody talks about the engineering tricks that make these monsters usable. Key-value caches, for instance. They speed up inference like a rocket. Without them, we’d all still be waiting for a single sentence to generate.
Is This Actually New Math?
No. Not really. The core ideas here—statistical modeling, vector spaces, matrix operations—aren’t fresh out of the oven. What’s new is the scale. The sheer volume of data. The complexity of the architectures. It’s like taking a simple recipe and making it feed a million people. The ingredients are the same, but the execution is… intense.
This whole LLM boom feels like a replay of the dot-com bubble, albeit with more tangible—though still abstract—results. Everyone’s scrambling to build the next big thing, promising revolutionary capabilities. But beneath the veneer of innovation, it’s often just smarter application of existing principles. The hype cycle is real, folks.
The text is encoded in the LLM’s vector space as token IDs, each token being a text fragment that has some probability of following another ID, such as when cats may be found on desks, as in the above photo by [Giles]. With inference multiple of such IDs are retrieved in a vector from which in successive steps a sentence can be pieced together.
This isn’t AI wizardry. It’s applied mathematics. And frankly, understanding that is more empowering than believing in a digital genie.
Why Should You Care About LLM Math?
Because the narrative needs correcting. We’re being sold a bill of goods that these are alien intelligences. They’re not. They’re incredibly sophisticated tools built on decades of research. Knowing the math helps you see the limitations. It helps you ask better questions about their capabilities and their potential for bias. It’s the difference between awe and understanding.
When you understand that an LLM is predicting the next most probable token, you stop expecting it to have opinions or feelings. You see it as a pattern-matching engine. A very, very good one. But an engine nonetheless.
It’s a machine for probability. And probabilities, while powerful, are not consciousness. Let’s not confuse the two. The math behind LLMs is fascinating, but its real value is in explain the technology. It’s about bringing us back to earth.
