#deep-learning

Maths for LLMs

Start with one input, one target, and one adjustable weight. Calculate a derivative by hand, apply a gradient-descent update, and verify every number in PyTorch.

• #deep-learning
• #gradient-descent
• #gradients
• #maths-for-llms
• #pytorch

LLMs, the whole thing · Part 10

Chapter 4 is the moment attention becomes a complete GPT. I build the model one inspectable component at a time, count its parameters, and trace the gloriously untrained first token.

• #deep-learning
• #gpt
• #llm
• #pytorch
• #transformers

Maths for LLMs

ReLU cuts every negative activation to zero. GELU computes a smooth tanh-shaped gate; I unpack the formula one real number at a time and place it inside GPT's feed-forward network.

• #deep-learning
• #gelu
• #gpt
• #maths-for-llms
• #pytorch
• #relu

Maths for LLMs

I refused to let LayerNorm remain a formula. Starting with four integers, I work through mean, variance, standard deviation, normalization and the exact PyTorch operation used for each GPT token.

• #deep-learning
• #layer-normalization
• #maths-for-llms
• #pytorch

LLMs, the whole thing · Part 9

Part 9 of LLMs, the whole thing: two heads, one token, and the difference between computing attention and organizing its tensors. A concrete PyTorch walkthrough of view, transpose, concatenation and the output projection.

• #attention
• #deep-learning
• #llm
• #multi-head-attention
• #pytorch
• #transformers

LLMs, the whole thing · Part 8

Section 3.5 of Raschka's book on seven real GPT-2 tokens: why masking after softmax breaks the row sums, why -inf on the scores fixes it in one pass, and how dropout is a second, random mask that doubles its survivors.

• #attention
• #causal-attention
• #deep-learning
• #dropout
• #llm
• #pytorch
• #transformers

LLMs, the whole thing · Part 7

Section 3.4 of Raschka's book on six tokens with real PyTorch output: three shared learned matrices, eighteen private q/k/v vectors, why d_in is forced, why scores are divided by the square root of d_k, and why 'weights' now means two different things.

• #attention
• #deep-learning
• #llm
• #pytorch
• #self-attention
• #transformers

LLMs, the whole thing · Part 6

The first attention mechanism in Raschka's LLM book, explained the way it finally clicked for me, a Java dev who bounces off dense maths: self-attention is just three operations, a dot product, a softmax, and a weighted average.

• #attention
• #deep-learning
• #llm
• #self-attention
• #transformers

LLMs, the whole thing · Part 2

Every modern LLM is a transformer, yet BERT, GPT and T5 behave very differently. The first article in a ground-up series on LLMs: the single distinction — the attention mask — that separates an encoder from a decoder.

• #bert
• #deep-learning
• #gpt
• #llm
• #transformers