#embeddings

Maths for LLMs

The positional-embedding sum quietly relies on broadcasting. Here is the whole rule — align from the right, stretch the 1s — and why PyTorch never copies a byte.

• #embeddings
• #pytorch
• #tensors
• #maths-for-llms
• #broadcasting

Maths for LLMs

The maths under positional embeddings, from zero: a token becomes what it is plus where it sits, and that 'plus' is a literal element-wise sum of two vectors. Why the differently-sized tables never actually clash, worked by hand, with slides.

• #embeddings
• #vectors
• #positional-encoding
• #maths-for-llms

LLMs, the whole thing · Part 5

The finale of chapter 2: how meaningless token IDs become the tensor attention actually consumes. It comes down to two lookup tables and one addition - a learnable embedding table for meaning, and a second, position-indexed table for order.

• #embeddings
• #llms
• #pytorch
• #positional-encoding

Working with Claude Code

I built a local instrument to look at embedding space, and most of what I assumed turned out to be wrong: nothing is orthogonal, no individual dimension means anything, and king minus man plus woman does not give you queen.

• #embeddings
• #nlp
• #dataviz
• #claude-code
• #llm

Maths for LLMs

A from-zero tour of the maths under large language models: how words become coordinates, why similarity is an angle, and the single multiply-and-add a computer uses to compare two meanings.

• #embeddings
• #cosine-similarity
• #maths-for-llms

LLMs, the whole thing · Part 1

The opening part of a series building LLMs from first principles. We start where every model starts: turning language into vectors and meaning into geometry, with a runnable script and its real numbers.

• #embeddings
• #fundamentals
• #machine-learning
• #nlp
• #llm