Hybrid Search: Combining BM25 and Vectors in Postgres

The pitch for vector search is that it understands meaning, not just keywords — ask about “canceling a subscription” and it’ll find the passage about “ending your plan” even though the words don’t match. That’s true, and it’s also the reason vector-only retrieval quietly fails on the queries that matter most in practice: the ones with a specific error code, a product SKU, a person’s name, or a config key. Embeddings are good at synonymy and bad at exactness, because the whole point of an embedding is to compress specific tokens into a fuzzy semantic neighborhood — which is precisely the property that makes ERR_429_RATE_LIMIT look almost the same as ERR_500_INTERNAL to a dense retriever.

The fix isn’t to abandon vector search — it’s to stop treating it as the only retrieval mechanism. BM25, the decades-old keyword ranking algorithm behind most full-text search engines, is exceptional at exactly what embeddings are weak at: exact term matches, rare tokens, identifiers, acronyms. Running both in parallel and merging the results — hybrid search — covers each method’s blind spot with the other’s strength. And critically, you don’t need Elasticsearch or a separate search cluster to do it: Postgres has had a capable full-text search engine built in since version 8.3, and with pgvector for embeddings, both retrieval paths can live in the same table, the same query, and the same transaction.

You’ll learn:

• Why dense-only vector retrieval systematically misses exact-term and identifier-style queries
• How Postgres’s built-in full-text search (tsvector, ts_rank, BM25-style ranking) actually works
• How to combine BM25 and vector scores using reciprocal rank fusion (RRF), with runnable SQL
• How to run both retrieval paths in a single Postgres query without a separate search infrastructure
• How to tune RRF’s k constant and per-source weights for your own query mix
• Where hybrid search still isn’t enough, and what to layer on top of it

Table of Contents

1. Why Vector-Only Retrieval Misses Exact Terms
2. BM25 and Full-Text Search in Postgres
3. Vector Search in the Same Database with pgvector
4. Reciprocal Rank Fusion: Merging Two Rankings Into One
5. RRF vs. Score Normalization
6. A Single Hybrid Query in Postgres
7. A Worked Example: One Query, Two Rankings
8. Tuning RRF: The k Constant and Source Weights
9. Where Hybrid Search Still Falls Short
10. Production Notes

Why Vector-Only Retrieval Misses Exact Terms

An embedding model maps text into a continuous vector space where semantically related concepts land close together. That’s a deliberate, useful compression — but compression means information gets thrown away, and what typically gets thrown away first is the exact surface form of rare tokens. Consider three real query types that break down under vector-only retrieval:

• Identifiers and codes: INVOICE-2026-004471, SKU-88213, a stack trace’s exception class name. These have no semantic neighborhood — the model has never seen this exact token enough times to have learned anything meaningful about it, so its embedding is essentially noise relative to similar-looking identifiers.
• Rare technical terms: a specific API method name, a config flag, a chemical compound. If the embedding model’s training data barely covered it, its vector position is unreliable.
• Negation and exact phrase matching: “do NOT use --force” and “always use --force” can embed as nearly identical vectors, because the surrounding words dominate the representation and short negation tokens contribute little to the overall vector.

BM25 doesn’t have these problems, because it isn’t trying to understand meaning at all — it scores documents by term frequency (how often a query term appears in a document) weighted against inverse document frequency (how rare that term is across the whole corpus), with length normalization so long documents don’t win purely by containing more words. A document containing the literal string INVOICE-2026-004471 will score highest for that exact query every time, deterministically, with no embedding model in the loop. This is exactly the class of failure covered in more depth in why RAG retrieves the wrong chunks — query-document asymmetry and embedding blind spots are one of the most common root causes of a retrieval pipeline that looks broken but is actually just missing this second retrieval path.

BM25 and Full-Text Search in Postgres

Postgres’s native full-text search isn’t literally the BM25 formula — it uses ts_rank / ts_rank_cd, which is BM25-adjacent (term frequency and coverage-based) but doesn’t include BM25’s inverse-document-frequency term or its specific length-normalization curve by default. For genuine BM25 scoring, the pg_search extension (via ParadeDB) or the vectorscale/pgvectorscale ecosystem provide it directly; for many hybrid search use cases, ts_rank_cd over a well-configured tsvector gets close enough that the distinction won’t matter unless you’re benchmarking search quality directly against a dedicated search engine. This post uses Postgres’s built-in full-text search because it requires zero extra infrastructure — the goal is a hybrid pipeline that lives entirely inside a normal Postgres instance.

Setting it up starts with a generated tsvector column and a GIN index over it:

ALTER TABLE documents ADD COLUMN search_vector tsvector
    GENERATED ALWAYS AS (to_tsvector('english', coalesce(body, ''))) STORED;

CREATE INDEX documents_search_idx ON documents USING GIN (search_vector);

Querying it with ranking:

SELECT id, body, ts_rank_cd(search_vector, query) AS bm25_score
FROM documents, plainto_tsquery('english', 'rate limit error') AS query
WHERE search_vector @@ query
ORDER BY bm25_score DESC
LIMIT 20;

to_tsvector handles stemming and stopword removal automatically (English “canceling,” “cancels,” and “canceled” all reduce to the same lexeme), which is part of why full-text search alone still recovers some of the synonymy that a naive substring match would miss — just not nearly as much as an embedding does for genuinely different phrasing.

Vector Search in the Same Database with pgvector

pgvector adds a native vector type and approximate nearest-neighbor indexing (IVFFlat or HNSW) directly to Postgres:

ALTER TABLE documents ADD COLUMN embedding vector(1536);

CREATE INDEX documents_embedding_idx ON documents
    USING hnsw (embedding vector_cosine_ops);

SELECT id, body, 1 - (embedding <=> :query_embedding) AS vector_score
FROM documents
ORDER BY embedding <=> :query_embedding
LIMIT 20;

The <=> operator is cosine distance; 1 - distance converts it to a similarity score for consistency with ts_rank_cd, which is already a “higher is better” score. Both indexes — GIN for full text, HNSW for vectors — coexist on the same table, which is the entire point: no data duplication into a second system, no keeping two databases in sync, no second point of failure.

Reciprocal Rank Fusion: Merging Two Rankings Into One

The hard part of hybrid search was never running two queries — it’s combining two incomparable scores into one ranking. BM25 scores and cosine similarity scores live on completely different scales and distributions; a BM25 score of 8.2 and a cosine similarity of 0.81 aren’t directly comparable, and naively averaging them lets whichever score happens to have a larger numeric range dominate the merge.

Reciprocal rank fusion sidesteps the scale problem entirely by throwing away the raw scores and using each result’s rank position instead:

RRF_score(doc) = Σ  1 / (k + rank_i(doc))

summed over each retrieval method i that returned doc, where rank_i(doc) is that document’s 1-indexed position in method i’s results, and k is a smoothing constant (60 is the standard default from the original RRF paper) that flattens the curve so the difference between rank 1 and rank 2 doesn’t dominate the score.

def reciprocal_rank_fusion(rankings: list[list[str]], k: int = 60) -> dict[str, float]:
    scores: dict[str, float] = {}
    for ranking in rankings:
        for rank, doc_id in enumerate(ranking, start=1):
            scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank)
    return dict(sorted(scores.items(), key=lambda kv: kv[1], reverse=True))

A document that ranks well in both BM25 and vector search accumulates a higher combined score than a document that ranks first in only one — which is exactly the desired behavior: strong agreement between two independent signals is more trustworthy than a single strong signal from one method alone.

RRF vs. Score Normalization

RRF isn’t the only way to merge two rankings, and it’s worth knowing what it gives up relative to the alternative. The other common approach is score normalization: min-max or z-score normalize each method’s raw scores onto a shared 0–1 range, then take a weighted sum of the normalized scores directly, instead of throwing the scores away and using rank position.

def minmax_normalize(scores: dict[str, float]) -> dict[str, float]:
    lo, hi = min(scores.values()), max(scores.values())
    span = (hi - lo) or 1.0
    return {doc_id: (s - lo) / span for doc_id, s in scores.items()}

Normalization keeps more information than RRF does — it preserves how much better the top result was, not just that it was first — which can matter when one method’s top result is a clear standout and the next is a distant second. The cost is fragility: BM25 scores in particular have a distribution that shifts with the corpus and the query’s term rarity, so a normalization range computed per-query can behave inconsistently across queries in ways that are hard to detect without close monitoring. RRF’s rank-only approach is less expressive but far more robust to exactly that kind of drift, which is why it’s the more common default for hybrid search in practice — reach for score normalization deliberately, when you’ve verified your score distributions are stable enough to justify it, not as a default.

A Single Hybrid Query in Postgres

Both retrieval paths and the fusion can be expressed as one SQL query using CTEs, so a single round trip to Postgres returns the final hybrid-ranked result set:

WITH bm25_results AS (
    SELECT id, ROW_NUMBER() OVER (ORDER BY ts_rank_cd(search_vector, query) DESC) AS rank
    FROM documents, plainto_tsquery('english', :query_text) AS query
    WHERE search_vector @@ query
    LIMIT 50
),
vector_results AS (
    SELECT id, ROW_NUMBER() OVER (ORDER BY embedding <=> :query_embedding) AS rank
    FROM documents
    ORDER BY embedding <=> :query_embedding
    LIMIT 50
),
fused AS (
    SELECT COALESCE(b.id, v.id) AS id,
           COALESCE(1.0 / (60 + b.rank), 0) + COALESCE(1.0 / (60 + v.rank), 0) AS rrf_score
    FROM bm25_results b
    FULL OUTER JOIN vector_results v ON b.id = v.id
)
SELECT d.id, d.body, f.rrf_score
FROM fused f
JOIN documents d ON d.id = f.id
ORDER BY f.rrf_score DESC
LIMIT 10;

The FULL OUTER JOIN is what makes this correct: a document that only BM25 found (because it contains an exact identifier no embedding would rank highly) still gets a non-zero RRF score and a chance to surface, and the same holds symmetrically for a document only vector search found. Widening each candidate set to 50 before fusing (rather than fusing only each method’s top 10) matters — a document that would have ranked 30th on vector similarity but 2nd on BM25 needs to actually be in the vector candidate pool for the fusion step to see it at all.

A Worked Example: One Query, Two Rankings

To make the merge concrete, here’s an illustrative trace (constructed by hand, not from a production system) for the query “rate limit error” against a small support-docs table:

Fusing at k = 60: doc_412 scores 1/(60+1) + 1/(60+9) = 0.0164 + 0.0145 = 0.0309; doc_88 scores 1/(60+14) + 1/(60+1) = 0.0135 + 0.0164 = 0.0299; doc_203 scores 1/(60+3) + 1/(60+4) = 0.0159 + 0.0156 = 0.0315. doc_203 edges out both single-method leaders because it ranked respectably on both signals — exactly the “agreement beats a single strong signal” behavior RRF is designed to produce. Notice also that neither doc_412 nor doc_88 would have reliably surfaced near the top under the other method alone; a vector-only pipeline buries the exact-match doc_412 at rank 9, and a BM25-only pipeline buries the semantically-relevant doc_88 at rank 14. That’s the concrete shape of the failure hybrid search exists to close.

Tuning RRF: The k Constant and Source Weights

The default k = 60 is a reasonable starting point, not a universal constant. Lower k values sharpen the curve, giving rank-1 results outsized influence over the fused score relative to rank-10 — useful if your top BM25 and vector results are usually genuinely better than the rest of the candidate pool. Higher k values flatten it, useful when rankings within each method are noisy and you don’t want the fusion to overreact to a single method’s top pick.

RRF as written above treats BM25 and vector search as equally trustworthy, which won’t be true for every query mix. A weighted variant lets you tilt the balance:

COALESCE(1.5 / (60 + b.rank), 0) + COALESCE(1.0 / (60 + v.rank), 0) AS rrf_score

A 1.5x weight on the BM25 term biases the fused ranking toward exact-match results — appropriate for a corpus with a lot of identifier and code-heavy content (API docs, log search, ticketing systems), where keyword precision usually matters more than semantic breadth. A corpus of long-form, conversational documents (support transcripts, knowledge-base articles) more often benefits from tilting the weight the other way. The only reliable way to pick these weights is the same eval-set methodology used for chunking strategy comparisons — build a query set from your own corpus, measure recall@k under a few weight combinations, and let the data decide rather than porting a default from someone else’s system.

Where Hybrid Search Still Falls Short

Hybrid search fixes the specific problem of vector search missing exact terms — it doesn’t fix problems that live upstream or downstream of retrieval. If chunk boundaries split an identifier from the sentence explaining what it means, BM25 can still find the chunk containing the identifier, but the LLM generating an answer from that chunk is working with the same incomplete context either way. Widening the candidate pool to 50 per method before fusing also means more candidates for a reranker to sort through if you’re using one — see the tradeoffs in reranking in RAG for when that extra step is worth its added latency on top of a hybrid-search candidate set. And hybrid search retrieves within whatever set of documents the query is scoped to — it says nothing about whether that scoping itself is correct, which matters a great deal in any system serving more than one tenant.

Production Notes

A few things that matter once this moves past a prototype:

• Rebuild the GIN index concurrently on large tables. CREATE INDEX CONCURRENTLY avoids locking writes during index creation, which matters once documents has more than a few hundred thousand rows.
• HNSW build time and memory scale with data volume. Set maintenance_work_mem generously during index creation, and budget real time for it on tables in the millions of rows — it’s not instantaneous the way a GIN rebuild on modest text volumes is.
• Cap candidate pool sizes deliberately. Fusing over the top 50 from each method (rather than 10 or 500) is a real tuning knob — too small and you miss cross-method disagreements that fusion is supposed to catch; too large and you’re paying for a vector scan and rank computation over candidates that will never surface.
• plainto_tsquery is the safe default for user input. It doesn’t support Postgres’s boolean query operators (&, |, !), which means it also can’t be broken by user input containing them — reach for websearch_to_tsquery only if you specifically want quoted-phrase and exclusion support exposed to end users.
• Re-run the eval set after every embedding model or tokenizer change. A hybrid pipeline has two moving parts that can each independently regress; changing one without re-measuring both scores risks quietly shifting the fusion balance you tuned.

Wrapping Up

Vector search and BM25 aren’t competing approaches to the same problem — they’re solving two different problems that happen to both get called “search.” One understands meaning and misses exact tokens; the other matches exact tokens and understands nothing about meaning. Reciprocal rank fusion is a genuinely simple way to combine them without needing to reconcile two incomparable scoring scales, and Postgres — with tsvector and pgvector living on the same table — means hybrid search doesn’t require standing up a second search cluster to get there. If your RAG pipeline is vector-only today, the fastest correctness win available isn’t a bigger embedding model — it’s giving BM25 a seat at the same table, literally.