Ranking the Whole Corpus

scoring and ranking documents

Part of: Semantic Search Engine

Cosine similarity scores one pair of vectors. A search engine scores a query against every document in the corpus, then orders them best-first. That's ranking. The ranking loop Score every document once, then sort. For a corpus of a few thousand documents this is fast: a handful of multiplications per document, no API call. The embedding call is the expensive part. Ranking is nearly free arithmetic. Ties need a rule Two documents can score identically, especially with toy or small-dimension vectors, or when two entries are near-duplicates. Python's sort is stable, but "stable" only means ties keep whatever order they arrived in, which is an accident of how you built the corpus rather than a decision. Give the search engine an explicit, deterministic tie-break. The simplest is that the smaller document ID wins: Sorting by (-score, id) sorts descending by score first (the negative sign flips an ascending sort into descending), then ascending by ID among ties only. Two searches on the same corpus and query now return results in the exact same order every time, which matters for testing, caching, and not confusing a user who searches the same thing twice. Why you rank before you trim Y

Challenge: Rank the Corpus