What to Know Before You Use an LLM

2026-07-13 Mon 02:15 KST · #llm · #aticaa · #dev · 한국어

There are plenty of articles explaining how LLMs work. Stories of changing code because of those fundamentals are rare.

This post is the latter kind. It’s a write-up of a Q&A with my AI partner on a Sunday night. I’m still learning, so I can’t use difficult words. But I remember exactly where I got stuck.

1. Terminology first — you can’t call what you can’t name

Q. The text I’m typing into this chat box right now — what’s it officially called?

A prompt; more precisely, this turn’s user message.

And the entire thing actually delivered to the model is called the context, or the context window. That these two are different was the first surprise.

Q. Different? It’s not just my message that gets sent?

No. Every turn, everything so far is delivered again.

When I send a one-liner like “oh, really?”, what the model receives is not that one line but:

[system prompt — the model's character and rules]
[project knowledge — attached files]
[conversation turn 1]
[conversation turn 2]
   ⋮  (everything so far)
[the one line I just typed]

It reads all of this and produces the next utterance. Every time. All of it. Again.

Q. So the model doesn’t actually remember the conversation?

Right. The model doesn’t continue a conversation — it’s closer to an actor who re-reads the entire script from the top every time and delivers the next line. Between turns, the model doesn’t exist.

This one fact explains almost everything that follows.

2. Tokens — the unit in which a model sees the world

Q. What’s a token? A character? A word?

Somewhere in between. Smaller than a word, bigger than a character.

The English word “understanding” splits into two pieces: under + standing (where exactly the cut lands varies by tokenizer). As a rough feel, 1–2 Korean characters or 0.75 English words ≈ 1 token.

Here’s what matters: the model doesn’t see text as characters. It sees a sequence of token numbers. Not “hello” but something like [45123, 8871].

Q. Why does that matter?

Because cost and limits are all denominated in tokens. API pricing is per-token, and the context window limit is a token count. As a conversation grows, the tokens to re-read every turn grow with it, and so does the bill.

So “write concisely” isn’t a matter of taste — it’s architectural hygiene.

3. Attention — the mechanism that decides what to look at

Q. Is attention roughly a kind of weighting?

Directionally, yes. Precisely, it’s the weighting of “when deciding the next token, which earlier tokens to look at, and how much.”

An example.

“I went down to the bank. A fish jumped in the water.” “I went down to the bank. I took out a loan.”

Same “bank,” different meanings. What split them? The surrounding tokens.

When processing each token, attention scans the other tokens in the text and piles weight onto the relevant ones. With “fish” and “water” near “bank,” the river side wins; with “loan,” the financial side does.

Computing how much every token relates to every other token — that’s attention, and it’s the heart of every LLM today.

4. Next-token prediction — but why does that become intelligence?

Q. In the end it’s just guessing the next token. Honestly, I can’t see how that alone makes all this possible.

Finding this strange is the correct response. Academia can’t fully explain it either.

But the intuition goes like this:

“To predict the next token well, you must in fact understand the world.”

Try completing these:

• “Water boils at 100 degrees, so it ___” → (you need physics)
• “The butler did it. He was the only one who ___” → (you need inference)
• “She smiled, but her eyes were ___” → (you need the contradiction of emotions)

Force “guess the next word” across the entire internet, trillions of times, and — as a byproduct of guessing well — a terrain map of grammar, facts, logic, causality, and emotion forms inside.

Prediction is the goal; understanding is the means it was forced to acquire to reach that goal. Like how truly acing an exam eventually requires truly knowing the material.

The way new abilities pop out as scale grows is called emergence.

Q. So is it understanding, or imitation?

An unfinished debate.

• Skeptics: “It’s a sophisticated parrot. Statistical mimicry.”
• Believers: “To imitate that well, there must be real representations inside. How is that different from understanding?”

There’s no verdict yet. The people wrestling with this question are currently drawing salaries at frontier labs.

5. Parameters — where is the knowledge stored?

Q. You said “a terrain map of emotion forms inside” — where and how is that stored?

In parameters. And a parameter is just a number.

A model is ultimately a giant table of numbers. When input tokens come in, layer upon layer of multiply-and-add runs against these numbers, and at the end out come the probabilities for the next token. The numbers used in that multiplication are the parameters — and there are hundreds of billions of them.

Training is the process of nudging those numbers, little by little. Make a prediction, and when it’s wrong, push all those billions of numbers slightly in the direction of being less wrong. Repeat trillions of times, and relations like “water ↔ boiling” get engraved as patterns in the numbers.

Q. So it’s like a DB row that says “row 3: water boils at 100°C”?

Not at all. That’s the most alien part of this architecture.

Knowledge is stored scattered and overlapping across hundreds of billions of numbers. A single parameter participates simultaneously in water’s boiling point, in cooking recipes, in the meter of a poem. Open up any one location and “water = 100°C” is nowhere to be found.

Which is why even the people who built it don’t fully know what’s inside. The field trying to read it out is called interpretability research — one of the frontlines of AI research today.

A thing humanity built without knowing its insides is now talking with hundreds of millions of people and writing code.

6. Where do the parameters “live”?

Q. Are those hundreds of billions of numbers floating in some giant memory?

Exactly. And this is what the global memory boom is actually about.

Parameters normally sit in a file on disk. But at inference time they must all be loaded into GPU memory (VRAM) — because generating each token requires sweeping through all of those numbers once.

Laptop RAM doesn’t come close. That’s why GPUs are ganged together, each holding a shard of one model.

Q. So the bottleneck is compute speed?

No. It’s the speed of hauling numbers from memory to the compute units.

Every token requires reading hundreds of billions of numbers, so the compute cores sit idle while memory bandwidth fails to keep up. This is the memory bandwidth bottleneck — and it’s also why high-bandwidth memory (HBM) is so expensive.

7. Quantization — what does compression cost?

Q. Quantization comes up constantly. If it’s compression, there must be a price — information loss or extra compute?

Good instinct. But the outcome is a little counterintuitive.

Quantization is the question of how many bits represent one parameter. It’s not a compression algorithm — it’s essentially rounding.

Store 0.3847291... as the nearest candidate value 0.375 — that kind of thing.

There is information loss. Every number gets slightly bent. But neural networks are trained to be robust to noise, and in a structure where hundreds of billions of numbers collaborate, individual numbers being slightly off rarely shakes the overall judgment.

Extra compute goes the other way — it gets faster. As we saw, the bottleneck is hauling numbers. Shrink them to a quarter and there’s a quarter as much to haul. The bandwidth win overwhelms the dequantization overhead.

Q. So it’s a free lunch?

No. Subtle abilities get shaved off first. Everyday conversation stays fine, while long reasoning chains and precise code generation start slipping, quietly.

Which is why practical intuition splits like this:

• INT8 / FP8: widely used in production. Loss is negligible.
• INT4: the standard for personal local inference. Great for everyday use; degradation is observable on precision work.
• Lower: experimental territory.

Not “quantization is bad,” but “how far to shave is decided by the use case” — that’s the correct sentence.

Q. Then are model tiers (like Haiku/Sonnet/Opus) also quantization?

No. This is an easy place to get confused — they’re two orthogonal dials.

A smaller-tier model isn’t a compressed big model — it’s a separate model, designed small and trained on its own. The brain capacity itself is different.

The distinction matters in practice: a quantized model “knows it but slightly fumbles,” while a small model “couldn’t hold it in the first place.” Different symptoms, different prescriptions.

8. So — the story of changing code because of the fundamentals

That’s the theory. Now one case where it actually changed code.

This happened while building an AI code-review system — a pipeline that gathers code chunks, assembles them into a prompt, and sends it to an API. One day we ripped out every use of set in that pipeline.

Python’s set is a fine data structure. It deduplicates, and lookups are fast. In a typical backend it’s often the right answer.

But in an LLM pipeline it can be a disaster. Everything above is exactly why.

First, the context is an ordered sequence of tokens. The model reads context linearly. The moment you put a set into a prompt, its arbitrarily-ordered serialization becomes the word order of the sentence the model reads. A set says “unordered”; the delivery medium says “order required.” It’s a type mismatch.

Second, prompt caching breaks. Caching only hits when the front of the prompt is byte-for-byte identical, token by token. If the order shuffles every run, identical content still misses the cache — and you’re billed in full, every time. A quiet leak.

Third, reproducibility dies. Pinning temperature to 0 doesn’t help. Different input, different output — as it should be. You review the same PR twice, get different results, and can no longer tell whether the cause is the model or the prompt.

Fourth, position changes meaning. Attention incorporates positional information. What comes early and what comes late carry different weight. When order wobbles, which code chunk gets the spotlight becomes a roulette wheel, and review quality variance grows.

So we set down one principle:

In an AI service, data structures are ordered by default, absent a specific reason otherwise.

It differs from ordinary backend common sense. An LLM pipeline is a world with different rules — output is sensitive to input order, and caching demands byte identity.

What stays with me

One scene from writing this post.

While explaining quantization, the AI said “you shouldn’t use quantized models in production.” I pushed back — “funny, nobody else seems to say that?” — and it corrected itself: in fact, a large share of production inference runs on quantized models.

Being plausibly wrong. That’s the nature of this architecture. A machine that picks the next token by probability finds it easier to emit “something plausible” than “I don’t know.”

Which is why knowing the fundamentals becomes armor. You learn what to doubt.

The record continues.