A field guide to vector similarity measures

Every retrieval system I’ve worked on in the last few years has, somewhere near its heart, the same tiny operation: take two vectors and produce a number that says how alike they are. Embed a query, embed a document, compare. That comparison - the similarity measure - is a one-line function, and yet it’s the thing people most often get subtly wrong, or reach for without quite remembering why.

So this is a refresher. Not a survey of exotic metrics, and not a benchmark - just the handful of measures you’ll actually meet, what each one geometrically means, and the small set of mistakes that keep resurfacing. The punchline, which I’ll spoil now and earn later, is that most of these measures collapse into the same ranking the moment your vectors are normalised. The interesting decision usually isn’t which measure - it’s whether you normalised at all.

The setup: similarity, distance, and three things you can measure#

First, a distinction that trips people up constantly. Some of these functions are similarities - bigger means more alike. Some are distances - bigger means less alike. Cosine similarity of 1 is a perfect match; Euclidean distance of 0 is a perfect match. Any time you switch measures, or feed a score into something that expects “higher is better,” you have to keep the sign straight. More on that in the gotchas.

Second, a framing I find useful. A vector really carries two independent pieces of information - and a third that is just those two combined:

• Magnitude - how long the vector is.
• Direction - where it points.
• Position - where the tip lands, which is simply magnitude and direction together.

Dot product cares about magnitude and direction. Cosine throws magnitude away and keeps only direction. Euclidean cares about position - the tip - so it feels magnitude and direction at once. Keep those words in mind and the whole zoo organises itself.

Dot product#

The dot product (or inner product) is the raw material everything else is built from:

$$\mathbf{a} \cdot \mathbf{b} = \sum_i a_i b_i = \lVert \mathbf{a} \rVert \, \lVert \mathbf{b} \rVert \cos\theta$$

Two ways to read the same number. The left form is mechanical: multiply componentwise, sum. The right form is geometric, and it’s the one worth internalising - the dot product is the length of a, times the length of b, times the cosine of the angle between them. Equivalently, it’s the projection of one vector onto the other, scaled by the other’s length.

That means the dot product moves for two different reasons: the vectors can point more alike (angle shrinks), or either vector can simply get longer. That magnitude-sensitivity is a feature or a bug depending on what you’re doing. When magnitude carries signal - some models let vector length encode “how much” of a concept is present, or term importance - you want it. When magnitude is just an artefact of, say, document length, it leaks in as a bias toward longer documents.

The dot product is also the workhorse inside more elaborate schemes. In ColBERT-style late interaction, the entire MaxSim operator is just a pile of dot products - each query token takes its maximum dot product against the document tokens, and those maxima are summed. No cosine, no distance; the similarity primitive there is the bare inner product.

Cosine similarity#

Cosine similarity is the dot product with magnitude divided out:

$$\cos\theta = \frac{\mathbf{a} \cdot \mathbf{b}}{\lVert \mathbf{a} \rVert \, \lVert \mathbf{b} \rVert}$$

By construction it lives in [−1, 1], and it depends only on the angle between the vectors - their direction - never their length.

This is why cosine is the default for semantic search. When you embed text, you usually care that two passages are about the same thing, not that one happens to produce a longer vector. Doubling a vector’s length shouldn’t make it a better or worse match for a query, and under cosine it doesn’t. This magnitude-blindness is exactly what makes the single dense vector so unforgiving when a document is a bag of many things - averaging a query concept in among two dozen others dilutes its direction, and no amount of length can rescue it. That failure mode is the whole subject of why single-vector search misses high-intent queries.

Here’s the fact that does most of the work in this article: cosine similarity is just the dot product of the normalised vectors. Normalise a and b to unit length first, and a · b is cos θ. So “dot vs cosine” is not really a choice between two measures - it’s a choice about whether normalisation happens inside the metric or beforehand.

Euclidean (L2) distance#

Euclidean distance is the straight-line, ruler distance between the two tips:

$$\lVert \mathbf{a} - \mathbf{b} \rVert = \sqrt{\sum_i (a_i - b_i)^2}$$

It’s a distance, so smaller is more alike, and it takes both magnitude and direction into account at once - two vectors can be far apart because they point different ways or because one is much longer. In practice engines usually rank by the squared distance, ‖a − b‖², and skip the square root: it’s monotonic, so dropping it changes the raw scores but never the order.

That sounds like a different animal from cosine, but on normalised vectors it isn’t. Expand the square for two unit vectors and the cross-term is exactly the dot product:

$$\lVert \mathbf{a} - \mathbf{b} \rVert^2 = \lVert \mathbf{a} \rVert^2 + \lVert \mathbf{b} \rVert^2 - 2(\mathbf{a} \cdot \mathbf{b}) = 2 - 2\cos\theta$$

So when everything lives on the unit sphere, Euclidean distance is a strictly decreasing function of cosine similarity. As the angle opens, the chord between the tips grows; as it closes, the chord shrinks. Different formula, same ordering.

The practical upshot: on normalised vectors, ranking by smallest L2 distance and ranking by largest cosine give you identical results. Approximate-nearest-neighbour indexes lean on this all the time - an engine can offer you “cosine” and implement it as normalise-then-L2 (or normalise-then-dot) under the hood, and the neighbours come back in the same order either way. (Strictly, only the documents need normalising for the ranking to hold - an unnormalised query rescales every candidate’s score by the same factor and so can’t reorder them - but normalising both sides is the simpler discipline.) The faster similar-document search in Elasticsearch 9.4 is a good example of the machinery that sits on top of exactly this equivalence.

Manhattan (L1) distance#

Manhattan distance - taxicab distance - sums the absolute per-dimension differences instead of the straight-line hypotenuse:

$$\lVert \mathbf{a} - \mathbf{b} \rVert_1 = \sum_i \lvert a_i - b_i \rvert$$

It’s the distance you’d walk on a city grid: no diagonals, only moves along the axes.

You’ll see it offered as an option in most vector databases, and there’s some (contested) evidence that it holds up better than L2 in very high-dimensional spaces, where distances concentrate and “everything looks roughly equidistant” - lower-order norms like L1 are said to degrade a little more gracefully. But for text embeddings it’s a niche choice - the models are almost never trained against it, and if you don’t have a specific reason to reach for L1, you probably want cosine or dot. I mention it mostly so it isn’t a mystery in the dropdown.

Hamming distance#

Hamming distance lives in a different world: binary vectors. It counts the positions at which two bit-strings differ, which on hardware is a XOR followed by a population count:

$$\operatorname{hamming}(\mathbf{a}, \mathbf{b}) = \operatorname{popcount}(\mathbf{a} \oplus \mathbf{b})$$

That operation is astonishingly cheap - a handful of instructions over a machine word, no floating-point multiplies at all - and that cheapness is the whole point. It’s what makes one-bit-per-dimension quantisation tractable. Squash each dimension of an embedding down to a single bit and you shrink the index 32x versus float32, and comparisons become popcounts. You lose fidelity, so binary Hamming is typically a first pass - cheaply rank a large candidate set, then rescore the survivors with a higher-precision measure.

Pure one-bit-both-sides Hamming is the clean case, but most production quantisation has moved past it. Asymmetric schemes keep one side coarser than the other and rank by an estimated dot product rather than a raw bit count - you can’t XOR a higher-precision query against binary documents. Two pieces in this vein are worth reading: what to make of TurboQuant, where the ranking metric stays a dot-product estimate even at one or two bits, and asymmetric query quantisation in DiskBBQ, where the asymmetry is in centroid granularity - the query is quantised once against a coarse parent centroid and reused across every document beneath it.

The big collapse#

Now I can pay off the promise from the top. For the common case - dense text embeddings, normalised to unit length - here’s what’s actually true:

• Cosine similarity is the dot product (normalisation is the only difference).
• Euclidean distance is a strictly decreasing function of that dot product.
• So all three produce the same ranking of candidates. Every one.

The measures differ in the scores they hand back - the raw numbers aren’t interchangeable - but the order they put your candidates in is identical. Which means the anxious question “should I use cosine or dot product or Euclidean?” mostly dissolves. If your vectors are normalised, pick whichever your engine makes fast and move on.

The question that does matter, and hides underneath all of this, is were the vectors normalised in the first place - and normalised consistently, the same way at index time and query time. That’s where the real bugs live.

Gotchas#

The mistakes, collected in one place:

• Forgetting to normalise, then using dot product. Dot product on un-normalised vectors is not cosine - magnitude leaks straight into the score, usually as a bias toward whatever produces longer vectors. If you mean cosine, either normalise first or use a cosine metric explicitly. Don’t half-do it.
• Normalising inconsistently. Unit-normalise your documents at index time but not your queries (or vice versa) and your “cosine” is neither cosine nor anything else meaningful. Whatever you do, do it identically on both sides.
• Comparing scores across measures - or across models. A cosine of 0.7 and an L2 of 0.7 mean nothing to each other. Worse, a cosine of 0.7 from one embedding model tells you almost nothing about a 0.7 from another; different models calibrate their similarity distributions completely differently. Similarity scores are only comparable within a single measure and a single model.
• Sign and direction confusion. Similarities go up with likeness; distances go down. Anything that ranks, thresholds, or feeds scores onward has to know which it’s holding. Flip it and you’ll cheerfully return the least relevant results - a bug that often passes a smoke test because the pipeline still runs.
• Negative dot products. Un-normalised inner products can be negative, and downstream code that assumes non-negative similarity (or takes a logarithm, or uses it as a weight) will misbehave. Cosine is bounded to [−1, 1]; the raw dot product has no fixed range.
• The zero vector breaks cosine. Cosine divides by ‖a‖ ‖b‖, so a zero-length vector - an empty or degenerate embedding - yields a divide-by-zero (NaN), not a similarity of zero. Screen out empty inputs before they reach the metric.
• Quantisation distortion. Hamming over binary vectors is an approximation of the real geometry. It’s brilliant for a cheap first pass, but the ordering it gives isn’t the ordering your full-precision measure would give - so rescore the top candidates before you trust the ranking.

So which do you use?#

The honest answer is: match the model. Almost every text-embedding model is trained with an objective built on cosine (or, equivalently, dot product over normalised vectors), and the cleanest thing you can do is use the measure the model was optimised for.

Concretely:

• Cosine - the safe default for semantic search. Right whenever direction is what matters and magnitude is noise.
• Dot product - use it when your vectors are already normalised, because it gives identical rankings to cosine while skipping the per-comparison division. Also the right choice on the rare model that deliberately encodes signal in magnitude - but only if it’s that kind of model.
• Euclidean / L2 - perfectly fine, and on normalised vectors interchangeable with the above for ranking. Reach for it when you’re thinking in terms of position, or when your index is tuned for it.
• Manhattan / L1 - a niche option; use it only with a specific reason.
• Hamming - for binary/quantised indexes where speed and footprint dominate, as a first pass ahead of a higher-precision rescore.

Five measures, and for the everyday case they mostly reduce to one decision made well: normalise your vectors, do it consistently, and pick whichever equivalent metric your stack runs fastest. The geometry is simpler than the dropdown of options makes it look.