Vector Databases Explained: How They Actually Work

Search a wine catalog for “wine for seafood” and a normal database will skip right past a bottle described as “pairs well with fish.” The words never line up, so the match never happens. The database is not broken. It compares strings, and those two strings are different, even though a person reads them as the same idea.

A vector database closes that gap. It has quietly become one of the most depended-on pieces of AI infrastructure, sitting behind ChatGPT answering questions about your own documents, Spotify finding a podcast from a vague description, and product recommendations that feel a little too good. This guide explains what a vector database is, why it exists, how embeddings and the HNSW index work, which tools to pick, and when you should reach for something simpler instead.

What is a vector database?

A vector database is a system built to store and search vectors: long lists of numbers that represent the meaning of unstructured data like text, images, or audio. Its core operation is similarity search. You hand it one query vector and ask a single question, which is “what is closest to this?” That is a different job from a traditional database, which is built to answer exact lookups like WHERE id = 123.

The reason this needs its own kind of database is that ordinary indexes fall apart in high dimensions. A B-tree works because numbers have an order you can split on. A 1,536-number vector has no natural order to sort by, so the database needs a different structure entirely to find near neighbors quickly. Searching by meaning turns out to be a geometry problem, not a string problem.

The problem: keyword search can’t see meaning

Keyword search matches strings, so it misses anything phrased differently. Search for “running shoes” and a pure keyword engine will not return “jogging sneakers” or “lightweight trainers,” because the literal words do not appear. Full-text engines like Elasticsearch soften this with tokenizing and ranking, but underneath they are still matching terms, not concepts.

This stopped being a niche problem. The large majority of data created today is unstructured: messages, photos, support tickets, audio. And people do not search the way a database schema expects. They type “something casual for a beach wedding,” which is a vibe, not a filter. Keyword systems fall apart on synonyms, phrasing, and context, and they fall apart faster as the data grows.

query "running shoes"

KEYWORD MATCH WHERE name LIKE '%running shoes%'

no row contains that exact string

✕ 0 results

VECTOR SEARCH

• ✓jogging sneakers
• ✓lightweight trainers
• ✓athletic footwear

◎ 3 strong matches

You cannot brute-force meaning with string matching. What you need is a system that measures how similar two things are, even when they share no words at all. That is exactly what a vector database is built to do, and it starts by changing what the data even looks like.

How vector databases actually work

A vector database works in four stages: embed, index, query, and filter. First you convert your data into vectors and store them. Then you build a special index over those vectors. At query time you convert the incoming request into a vector the same way, find its nearest neighbors, and apply any extra rules before returning results. Each stage exists to make the next one possible or fast.

01 Embed

Run raw text, images, or audio through a model. Out comes a vector: a list of 384 to 1,536 numbers.

→

02 Index

Build an ANN structure (usually HNSW) over millions of vectors so you never scan them all.

→

03 Query

Embed the incoming query with the same model, drop it in, ask for the nearest neighbors.

→

04 Filter

Apply metadata rules (date, permissions, category) and optionally rerank the top few results.

The one detail people get wrong is in stages one and three: you must embed your query with the same model you used to embed your data. Mix models, say index with OpenAI and query with a different provider, and the two live in incompatible spaces, so the distances become meaningless. Worse, nothing errors out. The system returns confident, wrong results, which is the kind of bug that takes days to spot.

What is an embedding?

An embedding is the vector a machine learning model produces to capture the meaning of an input. You feed a sentence like “the best coffee shops in Amsterdam” into an embedding model, and out comes an array of numbers, often 1,536 of them for OpenAI’s text-embedding models. Each number is one dimension. You do not know what any single dimension means, and you do not need to. What matters is distance.

The trick is that the model is trained so that similar meanings produce nearby vectors. “Coffee shops in Amsterdam” and “cafés in the Netherlands” end up close together, while “fix a Python import error” lands far away. Conceptual similarity becomes geometric proximity, and proximity is something a computer can measure in microseconds.

HNSW: how search stays fast at a billion vectors

Comparing your query to every stored vector is an O(N) operation. At a million vectors that might take ten milliseconds, which is fine. At a billion it is far too slow. So vector databases use approximate nearest neighbor (ANN) search, which gives up a sliver of accuracy for an enormous gain in speed. The industry-standard index is HNSW, published by Malkov and Yashunin in 2016.

Picture a multi-level highway system. The top layer has only a few nodes joined by long-range links, like interstates that move you across the country fast. Each layer down gets denser and more local, from state roads to city streets. A query enters at the top, hops to roughly the right area, then drops down layer by layer until it reaches the exact neighborhood. The result is logarithmic search time, sub-millisecond queries, at around 95% recall on typical workloads.

HNSW is not the only option. IVF splits the space into clusters and only searches the relevant ones. Product Quantization compresses vectors to shrink the memory bill. DiskANN keeps the index on fast SSDs for datasets too big for RAM. HNSW’s one real catch is cost: for peak speed the whole graph wants to live in memory, and at billion-scale that gets expensive fast.

Who actually uses vector databases?

The biggest use case by far is RAG, Retrieval-Augmented Generation. A language model on its own hallucinates and knows nothing about your private data. RAG fixes that by chunking your documents, embedding them, and storing them in a vector database. At query time it retrieves the most relevant chunks and feeds them to the model as context, so the answer is grounded in real facts instead of guesses.

The named examples are everywhere once you look. Perplexity built its retrieval layer on Vespa, serving 22 million users and around 780 million queries a month as of May 2025, fusing keyword, vector, and metadata signals in under 100 milliseconds. Spotify encodes podcast episodes and queries into a shared space so a search for “electric cars climate impact” surfaces an episode titled “EVs and the Environment.” Airbnb uses embedding-based retrieval trained with contrastive learning to narrow millions of listings into a small candidate pool before heavier ranking. And Vanguard reported its support search got over 12% more accurate after moving to hybrid retrieval on Pinecone.

Pinecone vs Milvus vs Weaviate vs Qdrant vs pgvector

The ecosystem splits into three groups. Purpose-built vector databases are designed from scratch for this and offer the most features at the cost of more to operate. Traditional databases with a bolted-on vector type let you add search next to data you already store. And libraries give you the raw algorithm with none of the database services around it.

Purpose-built

Designed from scratch for similarity search at scale. Most features, most ops to run.

• Pinecone
• Milvus
• Weaviate
• Qdrant
• Chroma

Vector-extended

Databases you already run, with a vector type bolted on. Often all you need.

• pgvector
• Elasticsearch
• MongoDB
• Redis

Libraries

The raw ANN engine. Fast, but no persistence, replication, or live updates.

• FAISS
• hnswlib
• ScaNN

For most teams the right answer is to start with what you already run. pgvector adds vector columns to PostgreSQL, keeps ACID transactions and joins, and handles tens of millions of vectors comfortably. Graduate to a dedicated system when you hit scale or feature limits. Here is how the main options compare.

One thing to know about FAISS, Meta’s open-source library from 2017: it is the engine inside many of these systems, but it is a library, not a database. No persistence, no replication, no live updates. It is the engine, not the car.

When should you not use a vector database?

Vector databases are not magic relevance machines, and a few honest limits decide whether you actually need one. The database does geometry, not language. It indexes whatever your embedding model produces, so if the model is weak, the results are weak, and no amount of database tuning will save you. Approximate also means approximate: at 95% recall you are missing 5% of true matches, which is fine for recommendations and a serious problem for legal or medical compliance.

On the model 100%

Search quality is capped by your embedding model. A weak model means weak results, and the database cannot fix that.

Recall, not 100 ~95%

Approximate search trades accuracy for speed. At 95% recall you miss 5% of true matches: fine for recommendations, risky for compliance.

Per billion vectors ~6 TB

A billion 1,536-dim float32 vectors need roughly 6 TB raw, before index overhead, and HNSW wants it in RAM.

Vector search also does not replace keyword search. For exact identifiers, classic keyword ranking is both more accurate and more explainable. That is why hybrid search, combining dense vectors with keyword scoring, is becoming the default in production rather than vectors alone.

So, do you actually need one?

If you are building anything that retrieves by meaning, semantic search, recommendations, or RAG on top of a language model, then yes, you need vector search. Whether you need a dedicated vector database is the more interesting question. For a lot of teams the answer is a vector type inside a database they already run, not a new service to operate and pay for.

The bigger shift is that vectors are turning into a data type rather than a separate product category. Every major database is adding native vector support, and the money is following: Databricks acquired Neon in 2025 and the overall market is projected to grow past 10 billion dollars by 2032. The mental model is what lasts: turn meaning into coordinates, and retrieval becomes geometry.

For the animated version of all of this, the embeddings, the HNSW walkthrough, and the trade-offs in about eleven minutes, watch the full explainer on YouTube.