Turning Text Into Numbers You Can Compare

Turn each chunk into an embedding vector and rank chunks by cosine similarity so search matches on meaning, not exact words.

Part of: Personal Notes Brain

Two sentences can mean almost the same thing while sharing barely a word. "I need to fix the login bug" and "auth is broken again" point at the same problem, and keyword search misses the match completely. To find notes by meaning, you turn every chunk into a vector of numbers, an embedding , and compare vectors instead of text. What an embedding is An embedding is a list of floats, often hundreds or thousands of them, produced by a model trained so that similar meanings land near each other in that number space. You call an embedding model once per chunk when indexing, and once per question at query time. You embed once and store the vector next to the chunk's text and metadata. A chunk that has not changed never gets re-embedded. That is the reason you build an index instead of embedding on every question. Comparing vectors with cosine similarity To measure how similar two vectors are, use cosine similarity , the cosine of the angle between them. It ranges from -1 for opposite meaning to 1 for identical direction. It ignores vector length and looks only at direction . That matters because a short question and a long chunk can point the same way in meaning-space even when their ra

Challenge: Rank Chunks by Similarity