Cosine similarity and the dot product

13 Aug 2026 · 6 min read

I wrote part one of this series assuming you were comfortable with dot products and cosine similarity. Someone read it and said, fairly: “I’m useless at maths, and none of that landed.” So this is the version I should have handed them first — the same ideas, built from zero, no maths background assumed. If you can multiply two numbers and add them, you already have everything you need.

The whole trick behind a model’s sense of meaning fits in one sentence: meaning becomes geometry. Words turn into positions in space, and “close in space” is engineered to mean “close in meaning.” Everything below is just unpacking that sentence.

1. Words become coordinates

A computer can’t read the word cat. It only does arithmetic on numbers. So the first move is to hand every word a list of numbers — a coordinate, a spot on a map. The part that makes it useful: the map is arranged so that words with similar meaning land near each other, and unrelated words land far apart.

A 2D map with food, animal and royalty words sitting in three separate clusters; a new word, kitten, arrows into the animal cluster. — Nobody places the dots by hand. The model learns an arrangement where similar meanings share a neighbourhood — so a new word like *kitten* lands next to *cat* and *puppy*.

Real encoders don’t use two dimensions like my drawing — they use hundreds (384 and 1536 are common). You can’t picture 384 axes, but the idea doesn’t change: it’s still a map, just with far more directions to spread meaning across.

2. Similarity is an angle, not a ruler

Here is the one upgrade that unlocks the rest. Draw each word as an arrow from the centre of the map out to its dot. Now you can ask a sharper question than “how far apart are they?” — you can ask “do these two arrows point the same way?”

Three cases: two arrows at a small angle labelled very similar, two at a right angle labelled unrelated, and two opposite arrows labelled contrary. — A small angle means similar meaning, a right angle means unrelated, opposite directions mean contrary. The angle is the whole signal.

Modern encoders stretch every arrow to the same length first — that’s what “L2-normalised” means, every vector pinned to a unit sphere. Once all arrows are the same length, length can no longer tell them apart, so direction is the only thing left that carries meaning. That is precisely why we measure the angle and ignore the distance.

3. Cosine turns the angle into one number

An angle in degrees is an awkward thing to compute with. We want a single number where bigger simply means more similar. Cosine is exactly that translator: give it the angle, it hands back a number between −1 and 1.

A number line from minus one to plus one, marked opposite, unrelated and identical, with cat-dog scoring high and cat-pizza near the middle. — 0° → +1 (identical), 90° → 0 (unrelated), 180° → −1 (opposite). This single number — cosine similarity — is what a vector database sorts on.

That’s the entire scale. It has a name, cosine similarity, and it is the number every retrieval system in the world ranks by.

4. The dot product: how a computer actually does it

A CPU doesn’t own a protractor — it never measures an angle. It has the two coordinate lists and nothing else. So how does it get a cosine out of them? With the cheapest operation there is: multiply the matching slots, then add the results. That is the dot product, start to finish.

The dot product of cat 0.6 0.8 and dog 0.8 0.6: multiply down each column to get 0.48 and 0.48, then add to 0.96. — Pair up the coordinates, multiply down each column, add the products: 0.48 + 0.48 = 0.96. A high score, so *cat* and *dog* come out very similar.

In code it’s a one-liner:

import numpy as np

# two unit-length embeddings (already normalised)
cat = np.array([0.6, 0.8])
dog = np.array([0.8, 0.6])

print(cat @ dog)                     # 0.96  -> very similar
print(cat @ np.array([-0.8, 0.6]))   # 0.0   -> unrelated (a right angle)

No trigonometry, no angle measured anywhere — just one multiply and one add per dimension.

5. Why the dot product is the cosine

This is the part I care about most, because it’s where the “scary formula” quietly dissolves. The textbook definition of the dot product is this:

The equation a dot b equals norm of a times norm of b times cosine theta, which collapses to a dot b equals cosine theta when both lengths are one. — The dot product equals the two lengths multiplied by the cosine of the angle. Normalise both arrows to length 1 and the lengths vanish, leaving the dot product equal to the cosine outright.

Read it in plain words: multiply-and-add equals (length of a) × (length of b) × (cosine of the angle). The whole formula is just bookkeeping for the two lengths. And once you’ve normalised every vector to length 1, those lengths are both 1 — they disappear, and you’re left with a · b = cos θ. That is the entire reason production systems normalise their embeddings: it makes similarity free — a single dot product, with nothing to divide afterwards.

6. Where the coordinates come from

One question I skipped: who decided that cat = [0.6, 0.8]? Nobody. The model invents every coordinate during training, through a process with a wonderfully blunt name.

A reference vector dog, with puppy being pulled closer and pizza being pushed apart, illustrating contrastive learning. — Contrastive learning: take a pair that should match and pull their vectors together; take unrelated things and push them apart. Repeat over billions of pairs and the map organises itself.

Start with random coordinates — pure noise, cat might sit next to pizza by accident. Then repeatedly show the model a pair that should be similar (a caption and its image, a question and its answer) and nudge those two vectors closer; take unrelated things and shove them apart. The objective has a name, InfoNCE, but the property that matters is that it’s relative: it only insists that related things score higher than unrelated ones, never that a pair hits a specific number. That’s also why you can’t compare a 0.8 from one model against a 0.8 from another — each score only ranks meaningfully inside its own space.

Why any of this matters

Because comparing two meanings collapses to one multiply-and-add over a couple of coordinate lists, a database can score a query against millions of stored vectors in milliseconds. The elegant geometry is why the number is meaningful; the embarrassingly cheap arithmetic is why it’s usable at scale. Every semantic search box and every RAG pipeline you’ve touched is, underneath, this one dot product run a few million times.

Next I’ll build on these vectors and get into attention — how a model lets words look at each other and shift their meaning in context. Same spirit as here: intuition first, symbols last.