Vector databases explained, and how to tell whether you need a dedicated one
Vector databases became the default answer to a question most teams had not finished asking. They are genuinely useful, and for a large share of products the database you already run will do the job.
By Quality AboveAll · · 8 min read
- A vector database stores embeddings and finds the nearest ones to a query vector, which is how semantic search returns results that share meaning rather than wording.
- Below roughly a million vectors, Postgres with pgvector is usually sufficient and removes an entire system from your architecture.
- Metadata filtering and permission scoping matter more in practice than raw query latency for most business applications.
What makes a vector database different
A traditional index answers "where is this exact value". A vector index answers "what is closest to this point". Text, images and audio are converted into embeddings, long lists of numbers positioned so that similar meanings land near each other, and the database's job is to find the nearest neighbours to a query point quickly.
Doing that exactly across millions of vectors is expensive, so these systems use approximate nearest neighbour algorithms that trade a small amount of recall for a very large speed gain. That trade-off is the core engineering decision: how much accuracy you are willing to give up for latency, and it is tunable rather than fixed.
When Postgres is genuinely enough
The pgvector extension adds vector types and indexes to PostgreSQL. For a corpus in the tens or hundreds of thousands of chunks, which describes most internal knowledge bases and product documentation sets, it performs well and brings a decisive advantage: your vectors live beside your relational data.
That co-location is worth more than most benchmarks suggest. Filtering results by tenant, user permission, document status or date becomes an ordinary SQL predicate rather than a synchronisation problem between two systems. Every separate store you add is another thing to back up, secure, monitor and keep consistent.
The cheapest vector database is the Postgres instance you are already paying for, backing up and monitoring.
When a dedicated vector store earns its place
Scale is the honest reason. Past a few million vectors, or when you need very high query throughput with tight latency budgets, purpose-built engines pull ahead because indexing and memory layout are their entire design goal rather than a bolted-on capability.
The second reason is operational specialisation: live index updates without rebuild pauses, native hybrid search combining keyword and vector scoring, and built-in re-ranking. If you need those and would otherwise build them yourself, buying is the cheaper path. That is the same reasoning we apply in build versus buy.
The features that actually matter in production
Metadata filtering quality is the one teams underestimate. A vector search that cannot efficiently restrict results to documents this user may see is not usable in a multi-tenant product, and bolting the filter on after retrieval quietly wrecks both relevance and result counts.
Re-indexing behaviour is the other. Embedding models get upgraded, and when they do every stored vector is obsolete because the new model puts meaning in different coordinates. Knowing in advance how you will re-embed and swap a corpus without downtime is the difference between an upgrade and an outage.
How to decide without over-engineering
Start with the data you have. Count the chunks you will actually store, not the number you might store in a hypothetical future, and be realistic about query volume. If both numbers are modest, use pgvector, ship the feature, and measure.
Design the retrieval layer behind a narrow interface so the store is swappable, then let real usage tell you whether you have outgrown it. This is the same API-first discipline that keeps any vendor decision reversible, and it beats picking a specialised database on day one for a workload you have not yet observed.
Frequently asked questions
Do I need a vector database for RAG?
You need somewhere to store and search embeddings, but that can be Postgres with pgvector. A dedicated vector database becomes worthwhile at higher scale or when you need native hybrid search and re-ranking.
How many vectors can pgvector handle?
In our experience it comfortably serves corpora in the hundreds of thousands of chunks with appropriate indexing. Past a few million, dedicated engines start to show a clear advantage.
What happens when I change embedding models?
Every existing vector becomes incomparable to new ones, so the whole corpus must be re-embedded. Plan a dual-index migration path before you launch, not after the model you use is deprecated.
Not sure whether your retrieval workload justifies another database? A free architecture review will size it against what you actually store and query.
