Tag
LLMs, the whole thing · Part 11
Chapter 5 connects next-token loss to real parameter updates, then separates training from decoding. My notes on batches, perplexity, validation, sampling, checkpoints, and pretrained GPT-2 weights.
LLMs, the whole thing · Part 10
Chapter 4 is the moment attention becomes a complete GPT. I build the model one inspectable component at a time, count its parameters, and trace the gloriously untrained first token.
ReLU cuts every negative activation to zero. GELU computes a smooth tanh-shaped gate; I unpack the formula one real number at a time and place it inside GPT's feed-forward network.
LLMs, the whole thing · Part 3
An LLM never sees your text — it sees integers. Building a tokenizer from a regex and a dictionary, hitting the out-of-vocabulary wall, and why GPT’s byte-pair encoding is the fix that needs no unknown token.
LLMs, the whole thing · Part 2
Every modern LLM is a transformer, yet BERT, GPT and T5 behave very differently. The first article in a ground-up series on LLMs: the single distinction — the attention mask — that separates an encoder from a decoder.