Lesson 1 - Why Vector Databases

Welcome to Why Vector Databases#

In Module 5 you built a working semantic search engine: embed every document, embed the query, compare the query to all of them, and return the closest. For fifteen FAQs that is instant and perfectly fine. But that approach hides a cost that grows with your data — every query re-scans every vector, and you have to keep all those vectors in memory and re-embed them whenever your program restarts. A vector database removes all three problems. It stores your embeddings on disk, builds an index so search doesn’t touch every vector, and lets you attach and filter on metadata.

This lesson is about why that matters. The next lessons get hands-on with Chroma.

By the end of this lesson, you will be able to:

• Explain why brute-force search doesn’t scale
• Describe what a vector database stores and indexes
• Explain what an approximate nearest-neighbor (ANN) index buys you
• Describe the “store once, query many” pattern and metadata filtering

You’ll just read and look at the architecture here — no setup required. Let’s begin.

The Problem: Brute Force Doesn’t Scale#

Your Module 5 search did one comparison per stored document, per query. That is a linear scan: with $N$ documents, each query costs work proportional to $N$. The table makes the trend concrete — these are rough orders of magnitude, but the direction is what matters:

And speed is only part of it. To compare against every vector you must hold every vector in memory, and because your Module 5 script embedded the corpus at startup, it had to re-embed everything every time it ran. Embedding is the slow, expensive step — repeating it on each launch is pure waste. At real-world sizes, “compare to everything, every time” simply falls over.

What a Vector Database Stores#

A vector database is purpose-built for exactly this job. At its core it stores three things together for each item:

• The embedding — the vector you learned to generate in Module 5.
• The original document — the text (or a reference to it) that the vector came from.
• Metadata — structured fields you attach, like topic, author, or date.

Storing the vector, the text, and the metadata together is what lets a single query return not just “the nearest vector” but the actual document it represents, optionally restricted to the metadata you care about. And because it all lives in the database, you embed each document once — not on every run.

What an Index Buys You#

The real magic is the index. Instead of comparing your query to every stored vector, a vector database organizes the vectors so it can jump almost straight to the closest ones. The most common technique is approximate nearest-neighbor (ANN) search: it trades a tiny, usually unnoticeable amount of accuracy for an enormous speed gain, finding the nearest neighbors without an exhaustive scan.

The payoff is a different scaling story. A linear scan grows with $N$; a good ANN index grows roughly with $\log N$. Going from a thousand to a million documents barely changes query time. That is the difference between a demo and a product.

Store Once, Query Many — and Filter#

Putting it together gives you a pattern that is the backbone of every production retrieval system:

1. Store once. Embed each document a single time and write the vector, text, and metadata to the database. It persists to disk, so a restart doesn’t lose or re-embed anything.
2. Query many. For each incoming question, embed just the query and ask the index for the nearest stored vectors. Fast, repeatable, cheap.
3. Filter by metadata. Restrict a search to vectors whose metadata matches a condition — “only documents where topic = shipping” — combining structured filters with semantic similarity.

That last point is something your Module 5 loop couldn’t do cleanly. Real applications constantly need “find the most relevant document that also belongs to this user / this category / this date range.” A vector database makes that a single call, which you’ll write in Lesson 3.

Practice Exercises#

Exercise 1: Count the work#

Your Module 5 search compared the query to every document. If a query takes 1 millisecond against 1,000 documents using a linear scan, roughly how long would the same approach take against 1,000,000 documents? Why is this a problem for a live application?

Exercise 2: Why store the text too?#

A vector database stores the embedding, the original document, and metadata together. Why isn’t the embedding alone enough — why keep the original text?

Exercise 3: Exact vs. approximate#

ANN search is “approximate” — it might not always return the single closest vector. Give one reason this trade-off is acceptable for semantic search, and one situation where you might want exact search instead.

Summary#

The brute-force search from Module 5 compares a query to every stored vector, which is fine for a handful of documents but scales linearly with $N$ — too slow, too memory-hungry, and it re-embeds everything on each run. A vector database fixes all three: it stores the embedding, the original text, and metadata together, persists them to disk, and builds an index (usually approximate nearest-neighbor) so search grows like $\log N$ instead of $N$. The result is the store-once, query-many pattern, with metadata filtering layered on top.

Key Concepts#

• Linear scan — comparing a query to every vector; cost grows with $N$.
• Vector database — a store for embeddings + documents + metadata, built for fast similarity search.
• Index / ANN — a structure enabling approximate nearest-neighbor search, roughly $\log N$ per query.
• Metadata filtering — restricting a similarity search to items matching structured conditions.
• Store once, query many — embed and persist data a single time; embed only the query thereafter.

Why This Matters#

Every production retrieval system — semantic search, recommendations, and the retrieval-augmented generation you’ll build next — runs on a vector database. Understanding what it stores and why the index matters is the difference between a toy that works on fifteen documents and a system that works on fifteen million.

Next Steps#

Continue Building Your Skills#

You now understand why a dedicated vector store exists and what its index buys you. Next you’ll use one for real — installing Chroma, creating a collection, and watching it store and search embeddings with just a few lines of Python.