Real Embeddings & Semantic Search

Semantic search

Part of: Embeddings & Semantic Search

Our movie vectors were hand-written. That does not scale to 10,000 movies or to free-form text like support tickets. For real text you call an embedding model : send it a string, get back a vector of floats. Then the exact same cosine-and-sort code you have been writing all module works on anything. What it is An embedding model converts text into a fixed-length vector of floats that represents its meaning. Semantic search is finding documents by meaning rather than exact words, using those vectors and cosine similarity. The two ideas are inseparable: the model gives you the coordinates, cosine ranks the closeness. How it works, and where keyword search breaks Old-school search matches literal words. Type "I forgot my password and can't sign in" and a keyword engine finds the doc containing "password." But it completely misses "My login isn't working anymore" because that doc shares zero words with your query. Same meaning, different vocabulary, and keyword search is blind to it. Embeddings fix this. A good model places "sign in," "login," and "log in" near each other in vector space because it learned from data that they mean the same thing. So semantic search retrieves the login

Challenge: Top-K Semantic Retrieval