How LLMs Actually Learn (LLMs 101, Part 1)

Let's start at the very beginning. A very good place to... fart. No that can't be right. Gotta train this model more.

Ask an LLM how many R’s are in “strawberry” and there’s a decent chance it’ll tell you two. Not three. Two. The most expensive, sophisticated, several-hundred-billion-parameter language model on the planet, the thing you’re paying twenty bucks a month to help you write code or summarize a PDF you don’t actually want to read, can’t always count letters in a three-syllable word.

The Count, trying to convince an LLM that Strawberry really does have 3 Rs

Which is hilarious, but it’s also telling you something pretty important about what’s happening inside the machine. The model isn’t reading the word the way you are. It can’t see the letters. It’s not even trying to. It’s looking at something completely different and then handing you words at the end as a kind of polite afterthought.

So that’s what this whole thing is about. How does this machine actually work, what’s it doing when it’s “learning,” and why does it sometimes flunk a kindergarten letter-counting exercise. Spoiler: the answer involves a lot of math and roughly zero understanding, and somehow that produces something that looks an awful lot like understanding anyway. Nobody knows why (Some people will pretend to though). We’ll get to that.

It’s All Math Wearing a Costume

To you, this sentence is a string of words. To the LLM, it’s a string of numbers. That’s it. Numbers in, math happens, numbers out, and then those output numbers get translated back into words for your benefit. The model never actually sees “strawberry.” It sees a token, which in a lot of models is something like the chunk “straw” plus the chunk “berry.” Two pieces. Possibly more (the exact split depends on the model and the word). Definitely not s-t-r-a-w-b-e-r-r-y.

So when you ask the model how many R’s are in strawberry, you’re asking a system that has no concept of letters to count something it can only see as two big lumps. That’s a little bit like asking someone who’s only ever heard a word spelled out in syllables to tell you how many of a specific letter are in it. They can guess. They can be right sometimes. They’re not actually counting, though, they’re approximating from context.

This is going to come up a lot in the series, this gap between what the model is doing internally and what we experience on the outside, so it’s worth exploring for a minute. The thing speaks English. It doesn’t know English. It knows numbers that, when you do enough math on them, come out the other side looking like English. That’s the whole game.

For an LLM, it’s math all the way down

Okay But How Does the Math Get Good At This

Right, so this is the actual training part. It’s surprisingly simple, conceptually. The complicated part is that you do the simple thing about a bazillion (ok, not a real number, but you get it... a LOT) times.

You start with a model that’s untrained. The internal numbers (people call these “weights,” but for our purposes you can think of them as a giant pile of dials, each one set to a random value at the beginning) don’t mean anything yet. If you ask this baby model to predict the next word in “the cat sat on the ___,” it’ll say something like “fork” or “blender” or possibly just garbage that isn’t even a word. Total nonsense. Garbage in, garbage out.

Now the trick. You take a massive pile of text. The entire internet, more or less, plus a lot of books, plus a lot of other stuff. You take a chunk of that text and you cover up the next word. You ask the model: what comes next?

The model guesses. The thing that makes the whole process work is that you already know the right answer (the next word is right there in the text, you just covered it up). So you can check. You can compare what the model said to what’s actually there.

Then you reach into the model and you nudge those millions or billions of dials. Just a tiny bit.

Then you do it again. Different chunk of text, different word covered up, model guesses, you check, you nudge.

Then you do it about a trillion more times... or you would if you were a timeless demonic being who somehow had millions of hands and near-infinite time. Most people working on LLMs probably don’t have THAT many arms, and time, so they use math. That bit IS complicated, but it’s also not actually that important to understanding the process, so for now let’s go with “the model is tested > MATH HAPPENS TO THE WEIGHTS > the model is tested again”. If the answer is “closer” mathematically to the token it SHOULD be predicting, the next change keeps going in roughly that direction... if not, if goes in a different direction.

Actual footage of tokens going through an LLM asked what the capital of Venezuela is or whatever.

That’s it. Predict, check, nudge. Predict, check, nudge. For months. On the beefiest computers on the planet. The dials slowly settle into configurations that make the model better and better at the prediction game. By the end of training, it’s really damn good at it. Give it the start of basically any sentence and it’ll cough up a plausible next word.

I want to be clear about how absurd this is. There’s no teacher in here. Nobody is explaining grammar. Nobody is showing it what a verb is. Nobody is teaching it that France is in Europe. The only feedback signal in the whole training process is “did you guess the next word correctly, here’s a tiny nudge based on the answer.” That’s the entire curriculum. Predict the next word. That’s what you’re getting graded on. Forever.

It Learns Things Anyway

This is where it gets weird, and I mean weird in the genuinely “this is fucked up, actually” way, not the marketing-copy “isn’t AI amazing” way.

To get really good at predicting the next word, the model has to “learn” a bunch of stuff that nobody asked it to learn.

An LLMs list of skills is… complicated.

BIG air quotes on “learn,” because there’s a lot of baggage there that does not apply in a practical way. Whether it’s finishing a sentence, writing a Python script, or taking the SATs, the model is still doing the same mathematical trick to the input tokens. Same process. Different context. Somehow that ends up looking like real understanding of pretty complex concepts... and THAT is fuckin’ weird.

To accurately guess the next word in “the capital of France is ,” the model has to actually have somehow internalized that France’s capital is Paris. To finish “she opened the door and saw a ,” it has to have absorbed enough about how stories work to predict something story-shaped instead of something random. To complete “the function returns __” in a piece of code, it has to have picked up something about programming, but... it hasn’t. It’s still just doing the SAME mathematical process to the input tokens.

So all these “skills” just fall out of the prediction game as a side effect. You’re *not training the model to know facts. You’re not training it to reason. You’re not training it to write code or translate languages or summarize documents. You’re training it to be really good at one specific task (predict the next word in arbitrary text) and along the way it kind of has to develop competence at all this other stuff, because otherwise it would be bad at the task.

That’s the part that should make your head tilt a little. Nothing in the training process is shaped like “learn things.” It’s all shaped like “play the prediction game.” Out the other end falls a thing that knows things, or behaves enough like a thing that knows things that the difference is hard to even measure.

Now For The Embarrassing Part

So you’ve got the basic story. Predict-check-nudge a trillion times, model gets good at the game, useful skills fall out as a happy accident. Cool. Why does this WORK though?

We don’t have a fucking clue.

There’s not some theory we can point at that says “given a model of this size, trained on this much data, with this much compute, here are the skills it’ll have.” We can sort of predict that it’ll get better, because other models have gotten better, and the bigger the model, the better they generally get at a lot of the skills that tend to fall out. We can tell you the loss number will go down (loss is just “how wrong the model is on average”). What we can’t tell you is which specific abilities will pop into existence at which scale. People keep being surprised. They make a model 10x bigger and it can suddenly do something the previous one couldn’t, and the people who built it find out about it the same way you do, by trying things and going “huh, look at that.” Like Christmas morning, except they’re the ones who wrapped the presents and they don’t remember what they put in the boxes.

There’s a partial answer that goes by the name “compression.” The idea is that to predict text well across a giant pile of every possible kind of text, you have to internalize the structure of the world that produced the text. Grammar. Facts. Reasoning patterns. Storytelling. Code. All of that stuff constrains what the next word probably is. If you compress a trillion words of human writing into a model with a finite number of dials, the most efficient compression looks a lot like understanding.

This is more like hand-wavey bs than an answer, if I’m being real about it. It tells you why something LIKE intelligence MIGHT falls out, in roughly the same way it tells you why squeezing a balloon makes it bulge somewhere. It doesn’t tell you why this particular squeezing process makes THIS particular bulge appear in THIS particular spot. The mechanism is still mostly opaque. It also does absolutely nothing to explain why these skills fall out instead of the thing just not getting good at the prediction game for things outside of the training data. Which is... awkward, imo.

There are people working on opening the models up and looking inside, an area called mechanistic interpretability. They’re finding actual little circuits in there, things that do specific computations, and that work matters a lot. “We found a circuit that does X” is not the same as “we understand why training reliably builds circuits,” though. Not yet. Maybe ever, who knows. It’s also probably not AMAZING that people have spent millions (probably billions at this point, but it’s hard to break down accurately) SPECIFICALLY on trying to understand the whole WHY of it... with not really much luck so far other than some very vague theories and some sort of semi-consistent ideas about circuits within the model which map out to vague concepts more often than specific skills and is very much still a guessing game where they can’t even point at the pieces with any confidence, let alone understand how they work together to produce actual skill and understanding.

To be clear, people do understand the MECHANICS of the training process. They know how to calculate loss pretty consistently. They know how to run backpropagation in ways that show literal improvements to the models. They know that, historically, bigger models trained on more data and compute tend to get better in fairly predictable-looking ways.

AI companies figuring out that the only thing they KNOW about how LLMs “learn” is “bigger means smarter” and rushing to make the biggest model they can.

But that is not at ALL the same as understanding why this works. It is closer to noticing that every time you pour more fuel into the weird machine, it gets louder, faster, and occasionally learns Spanish. Useful? Absolutely. Reassuring? Not really.

So this is where we are. We have a process that works incredibly well, that we cannot really explain, that produces something that behaves intelligently for reasons we cannot fully derive from first principles, and we are spending hundreds of billions of dollars on the assumption that if we just keep doing it bigger it’ll keep working (which so far HAS been true... to be fair). Which is more than slightly wild. Fingers crossed I guess.

“It’s Just Autocomplete”

So far, everything I just described, the predict-check-nudge a trillion times thing, that’s called pretraining, and it gets you a system that’s freakishly good at completing text. Give it the start of a sentence, it’ll finish the sentence. Give it the start of a paragraph, it’ll finish the paragraph. Give it a half-written sonnet about the misery of choosing a parking spot at Costco (a perfectly serious topic in my opinion), it’ll go ahead and write the rest.

That’s not a chatbot. That’s a really, really good autocomplete. Which is where a lot of the “it’s just autocomplete” stuff comes from, since... it IS that... but it’s not JUST that.

The thing that turns the autocomplete into something that politely answers your questions instead of just continuing whatever you typed at it is a whole separate phase that happens AFTER pretraining. That’s where humans get involved, where the model learns to follow instructions instead of just plowing forward, where it gets the personality and the safety training and the willingness to say “I don’t know” instead of confidently inventing an answer.

Which is part two.

Next time we crack the box open and watch humans try to beat politeness and honesty into several hundred billion dials with a technique that is basically digital dog training at planetary scale. Bring snacks.