Hybrid Search
Layered Search Ranking started as keyword searching. Semantic Search then brought vector embeddings to the same product catalog. Each one wins on a different kind of query.
BM25 is good when the query and doc have matching terms. It knows the query walnut record cabinet cares about seeing the words walnut, record, and cabinet. Embeddings come into their own when the terms don’t match. A user could search for vinyl storage console and mean our record cabinet, even if that exact phrase never shows up in the title.
Hybrid search runs both searches and then merges the results together into one result set. The art comes in merging them together without one scoring model trumping the other.
It looks something like this:
flowchart TD
Q["<strong>Query</strong><br/><span class='mermaid-detail'>vinyl storage console</span>"]
L["<strong>BM25</strong>"]
V["<strong>Vector</strong>"]
LR["<strong>BM25 ranks</strong><br/><span class='mermaid-detail'>1. vinyl_record_cabinet<br/>2. oak_record_stand</span>"]
VR["<strong>Vector ranks</strong><br/><span class='mermaid-detail'>1. walnut_media_console<br/>2. vinyl_record_cabinet</span>"]
RRF["<strong>RRF</strong>"]
F["<strong>Fused ranks</strong><br/><span class='mermaid-detail'>1. vinyl_record_cabinet<br/>2. walnut_media_console</span>"]
Q --> L
Q --> V
L --> LR
V --> VR
LR --> RRF
VR --> RRF
RRF --> F
Mismatched scores
BM25 produces a score from term frequency, inverse document frequency, field length normalization, field boosts and any query structure around a text match. A vector search produces a score from an embedding model, a distance function, and the structure of your vector index. Both scores can be represented by numbers. However, you cannot directly compare a BM25 score to a vector score.
You can normalize the scores and combine them, however. OpenSearch has enabled that through its normalization processor for quite some time now. The challenge is that the normalization needs to understand the distribution of scores per query.
For example, if one document is light years ahead of the rest, normalization will compress everyone else towards zero. If every document scores about the same, normalization will stretch those small differences to cover the entire range. Vector scores suffer from this same problem.
Reciprocal rank fusion elegantly sidesteps this problem. Instead of arguing over whether a BM25 score of 14.2 is better than a cosine score of 0.78, RRF just asks: at what position did this document rank in each list?
Reciprocal rank fusion
Reciprocal rank fusion gives each document a tiny fraction of points from each ranked list it appears on:
rrf_score(document) =
1 / (rank_constant + bm25_rank(document)) +
1 / (rank_constant + vector_rank(document))
If a document is absent from a list, it receives no contribution from that list. The highest-ranked document in each list is rank 1, so with the default rank_constant of 60 it contributes 1 / 61. Using a smaller value weights the very top of each list more. Larger values flatten the curve, making rank positions feel more equivalent.
Here’s a small example:
document bm25 rank vector rank rrf score
vinyl_record_cabinet 1 2 0.0325
walnut_media_console 4 1 0.0320
oak_record_stand 2 - 0.0161
The first two documents scored closest because both lists ranked them highly. The third scored well for BM25, but only one list rewarded it.
OpenSearch just released a writeup on introducing RRF for hybrid search. Their implementation uses the score-ranker-processor, which fires between the query and fetch phases and blends your query clause rankings before the final hits are returned.
The pipeline
Build a search pipeline with a score-ranker-processor, with the combination technique set to rrf:
PUT /_search/pipeline/products-rrf
{
"description": "Rank fusion for product hybrid search",
"phase_results_processors": [
{
"score-ranker-processor": {
"combination": {
"technique": "rrf",
"rank_constant": 60
}
}
}
]
}
I specified rank_constant even though 60 is the default. I prefer to make it explicit since it’s easier to look back at your pipeline later.
Now execute a hybrid query against it. OpenSearch executes each clause independently, then merges the resulting lists through the pipeline using RRF.
The lexical branch can largely mimic the BM25 query in Layered Search Ranking. The vector branch searches through the embedding field discussed in Semantic Search:
POST /products_semantic/_search?search_pipeline=products-rrf
{
"size": 10,
"_source": {
"excludes": ["search_embedding"]
},
"query": {
"hybrid": {
"queries": [
{
"bool": {
"must": [
{
"multi_match": {
"query": "vinyl storage console",
"fields": ["title^4", "description"],
"type": "best_fields"
}
}
],
"filter": [
{
"range": {
"stock_available": {
"gt": 0
}
}
}
]
}
},
{
"knn": {
"search_embedding": {
"vector": [0.012, -0.034, 0.008, ...],
"k": 100,
"filter": {
"range": {
"stock_available": {
"gt": 0
}
}
}
}
}
}
]
}
}
}
Create the query vector using the same model you used to create the index, adding a query prefix if your model requires one.
Our hybrid query supports up to five clauses, but let’s use two clauses to start: a lexical and a vector clause. This should be enough for us to tell if hybrid search is useful or not.
Notice how our stock filter clause shows up twice. This is intentional. If the vector branch were the only one filtering out-of-stock items, then RRF would still give matching credit to items the lexical branch returned, and those items would appear on the page.
Tuning
RRF has fewer parameters than score normalization, but you might still want to tune what’s there.
Start with candidate depth. Recall that RRF can only fuse documents returned by each branch. If the vector branch uses k = 10 and your final page is size = 10, then RRF has no depth to perform any fusion. This is why we used k = 100 in the example above. Experiment by pulling a deeper vector candidate set, measuring latency, and choosing how much recall you can afford.
Next, tweak rank_constant. It’s also fine to leave the default 60 here, as this value shouldn’t harshly penalize low-ranked documents while still boosting documents appearing near the top of both lists.
Most of your work from here will be tuning branch quality. Remember that the RRF processor doesn’t allow you to individually weight branches. If your lexical branch is too weak, tune your multi_match query, field boosts, analyzers, and exact fields. If your vector branch is too noisy, tune your embedding text, model, k, and filters. Resist the urge to simulate a BM25 vs vector weight by fiddling with rank_constant, as that will change the rank curve for all your branches.
Failure modes
RRF doesn’t recover from bad inputs. If you give it bad input such that BM25 returns matches that are way too broad and the vector index returns way too loose semantic neighbors, fusion can still output an average-at-best page.
Exact matches are the worst culprit here. Part numbers, SKUs, etc. are generally only going to lexically match. If your lexical branch doesn’t rank them highly enough, you shouldn’t expect your vector branch to rescue you. For known-item queries, I would either weight your exact clauses extremely heavily or divert queries out of hybrid search entirely if the user obviously wants an exact match.
The second failure mode is false agreement. Two branches can vote for the same bad document for completely different reasons. A product page might match the query text because it happens to include stock marketing copy. It might also land near the query vector by random chance. RRF will reward that overlap. Overall metrics can conceal it. Inspect the junk results for each query class instead.
Evaluation
Pick judged queries from Offline Ranking Metrics and score all three configurations side by side:
bm25
vector
hybrid_rrf
Segment results by query class. Exact product name queries will trend differently than general category queries or vocab-mismatch queries, so observing these classes individually allows you to see where fusion adds value.
RRF achieved ~4% lower NDCG@10 than score-normalized hybrid search in OpenSearch’s launch benchmark, but was slightly faster. That’s about what I would have expected: RRF’s simplicity makes it appealing as a first-pass hybrid baseline since you don’t need calibration, but a carefully tuned normalization pipeline can still beat it.
When you roll it out, pay attention to your online ranking signals. The success metrics will be similar to when you built the vector index: fewer empty pages and fewer bounces on vocab-mismatch queries. The biggest new risk will be fusion surfacing something that displays OK but links to the wrong product, so weight success rate more heavily than CTR. If those numbers stop improving, that’s when it’s worth reaching for heavier normalization or learned ranking.