Measuring Similarity with Cosine

cosine similarity

Part of: Semantic Search Engine

You have vectors. The next question: given a query vector, how "close" is it to a document's vector? For embeddings the standard answer is cosine similarity . Why not just measure distance? The obvious first move is straight-line distance between two vectors. Embedding vectors vary in length (magnitude) for reasons that have nothing to do with meaning. A longer document sometimes produces a vector with larger magnitude than a short one, purely as an artifact of the model. Measure raw distance and a long document about the exact right topic can score worse than a short, only vaguely related one, just because of its magnitude. What cosine similarity measures instead Cosine similarity ignores magnitude and measures the angle between two vectors. Two vectors pointing the same direction score 1.0, no matter how long either one is. Perpendicular vectors score 0.0. Opposite directions score -1.0. Because only direction counts, doubling a vector's length leaves its similarity to everything unchanged, which is what you want when comparing meaning across documents of different lengths. The formula - Dot product (dot): multiply matching positions and sum them. It's large when two vectors poin

Challenge: Cosine Similarity from Scratch