Hybrid Search: Combining Keyword and Semantic Retrieval in One Ranking

Why not just go fully semantic

Semantic search is excellent at descriptive and intent-driven queries — "something warm for a toddler at the bus stop" — and weaker exactly where keyword search is strong: an exact SKU, a part number, a model code, a brand token typed on its own. Hybrid search runs both retrievers and fuses their results into one ranking, so the query "SKU-4471" matches exactly while "warm toddler coat" still works.

The two retrievers

• Lexical retrieval. An inverted index (BM25-style) or its equivalent, returning candidates by term match. Fast, exact, and unbeatable on identifiers and rare tokens.
• Semantic retrieval. A vector index (approximate nearest neighbour) returning candidates by meaning proximity. Handles paraphrase, synonyms, and descriptive phrasing without a hand-maintained list.

You run both for every query and end up with two candidate lists that overlap partially and rank things differently.

Fusing the two lists

Two common approaches:

Weighted score fusion. Normalise each retriever's scores to a comparable range, then combine with a weight — alpha × semantic + (1 - alpha) × lexical. Flexible, but sensitive to score distributions that differ between retrievers and shift as the index changes.

Reciprocal rank fusion (RRF). Combine by rank position, not score: each document gets 1 / (k + rank) from each list it appears in, summed. It ignores raw scores entirely, so there's nothing to normalise and nothing to drift. It's the robust default, and it's what most teams reach for first.

Then re-rank

Fusion picks a good candidate set; it doesn't produce the final order. The fused top-N goes into a cross-encoder re-ranker that scores each query-product pair jointly and returns the final top-K. On our semantic search project the pipeline is retrieve then re-rank then serve — a hybrid version adds the lexical retriever alongside the vector one before that re-ranking step.

A concrete pipeline shape

For a single query: the lexical retriever returns its top 50-100 by term match, the vector retriever returns its top 50-100 by embedding proximity, both run in parallel. Reciprocal rank fusion merges the two lists into one ranked set. That fused set — still fairly broad — goes into the cross-encoder re-ranker, which scores each query-product pair and returns the final top 10 shown to the shopper. Two retrieval calls, one fusion step, one re-rank call. Both indexes can live in the same database, so "hybrid" doesn't mean a second system to operate — it means a second retriever reading from the same store.

Tuning it

• The weight or the routing. Either tune a global alpha, or route per query: an all-caps alphanumeric token leans lexical, a full sentence leans semantic. A lightweight query classifier does this well.
• Candidate depth. How many results each retriever contributes before fusion — too few and fusion can't recover a good match one retriever missed; too many and you pay latency for candidates that never rank.

What hybrid fixes, concretely

• SKU-4471 retrieves that exact product instead of "similar" ones.
• A brand name typed alone ranks that brand's products, not lookalikes.
• A misspelled model number still matches via the lexical side's fuzzy matching.
• A descriptive, natural-language query still works via the semantic side.

Where this stops being right

• A catalogue with no meaningful exact-identifier queries. If nobody searches SKUs or part numbers, pure semantic retrieval may be enough — skip the lexical index and the fusion step.
• A tiny catalogue. Keyword search plus a synonym list clears the bar without any of this.
• A very tight latency floor. Two retrievers plus fusion plus re-ranking is more hops; type-ahead under ~150ms may force a simpler design.

FAQ

RRF or weighted score fusion? RRF is the safer default — there's no score normalisation to get wrong and it doesn't drift as the index changes. Reach for weighted fusion only if you need fine control and can maintain it.

Do I still need re-ranking with hybrid? Yes. Fusion selects candidates; re-ranking orders the final top-K by joint query-product relevance. They're different jobs.

Does hybrid double my infrastructure? It adds a lexical index alongside the vector one — modest, and both can live in the same database. The extra cost is the two retrieval calls per query and the fusion step.

ISTRALLEN builds hybrid retrieval that keeps exact-match precision alongside semantic recall; see AI for Retail.