What you will be able to do
- Explain why a reproducible evaluation must come before any change to an AI Search configuration
- Pick recall@k or precision@k to match what a retrieval failure costs the use case
- Read the AI Search retrieval quality evaluation dashboard and its relevance metrics
- Map a result pattern (for example, full-text beating ANN) to the configuration that causes it
Key concept
Strategy comparison as diagnosis — To find the cause of poor retrieval in AI Search, run the same query set against every retrieval strategy, with and without the reranker, and compare the scores. The strategy that wins, or the fact that none of them does, points to the setting (or the data) at fault.
1.Measure before you change anything
A typical report says something like "the agent keeps answering from the wrong manual" or "it can't find error code E404". The tempting response is to change a setting straight away: turn on hybrid search, add a reranker, or re-chunk the documents. Databricks' retrieval quality guide says to wait. Before optimizing anything, you need a reproducible evaluation system. If you don't have one, set it up first, because without measurement you can't tell whether a change helped or hurt.
You also need to know what "bad" means for this agent. The guide divides use cases by which failure costs more. When recall matters most, for example in a RAG agent where missing context leads to hallucinations, or in compliance search, track Recall@k. When precision matters most, for example in entity resolution or a tech-support knowledge base where the exact fix must appear near the top, track Precision@k. Some cases, such as M&A due diligence, need both. Set a latency target as well: something like under 2 seconds to first token for a RAG agent, or under 100 msec for a search bar. Every fix you try later has to stay within that budget.
Checkpoint 1 of 5· Match them up
Match each use case to the metric the guide says to track first
Tap a term, then the definition that fits it.
Missing context makes a RAG agent answer wrongly, so recall matters most there. Entity matching needs only the right match. Prior-art search needs complete coverage with the most relevant patents ranked first.
“RAG agents: Missing key context leads to incorrect answers or hallucinations.”Source: docs.databricks.com
Sources1
2.The built-in retrieval quality evaluation
AI Search has a built-in retrieval quality evaluation (Beta) that runs this comparison for you. It requires a managed Delta Sync index. Start it with Evaluate search quality on the index page. No configuration is needed, because the defaults are filled in from the index metadata. The evaluation inherits Unity Catalog permissions from the index, so anyone who can query the index can run it. The run has four stages.
Checkpoint 2 of 5· Put it in order
Put the stages of the AI Search retrieval quality evaluation in order
- 1.Search across strategies: run each query with ANN, hybrid and full-text, with and without the reranker
- 2.Score relevance: an LLM judge rates every query-document pair
- 3.Compute metrics and analyze, with confidence intervals
- 4.Generate queries: sample documents and have an LLM write natural-language and keyword queries
Queries have to exist before they can be searched, and results have to be scored before metrics can be computed from them. The documentation lists the stages in this order.
“Score relevance: An LLM judge evaluates every query and retrieved document pair on a 4-point relevance scale.”Source: docs.databricks.com
The judge scores each result from 0 to 3: Highly Relevant, Relevant, Partially Relevant, or Not Relevant. A document in the wrong language scores 0 even if its content is correct. The dashboard's headline metric is DCG@10, shown next to the query type that achieved it. DCG uses the full graded scale and discounts results by position, so a 3 at rank 1 counts for much more than a 1 at rank 10. Unlike NDCG, it also measures how much useful content was returned in total. Two result sets can both have a perfect NDCG of 1.00 while one has a DCG almost twice as high, because every result in it was useful.
Other views on the dashboard help with diagnosis. Average relevance gives a quick snapshot. The relevance distribution shows its shape: a bimodal split of many 3s and many 0s suggests one particular query pattern is failing. MRR shows how soon the first relevant result appears. A failed-queries table lists every query whose top result scored 0.
Checkpoint 3 of 5· Check yourself
Two strategies have the same average relevance. One returns mostly 3s and 0s, the other mostly 2s. What does the first distribution suggest?
The documentation uses exactly this bimodal shape as an example of why you should look at the distribution and not only the average.
“a bimodal distribution (many 3s and many 0s) might suggest that a query pattern isn't being retrieved well and needs attention.”Source: docs.databricks.com
Sources2
3.From result pattern to root cause
To turn the comparison into a diagnosis, you need to know what each retrieval algorithm is good at. The query type is set per query with query_type, and it defaults to ann. An agent that was never explicitly configured is therefore running pure vector similarity, whatever its users actually type.
| query_type | How it works | Underperforms when |
|---|---|---|
| ANN (default) | Vector embeddings find semantically similar documents | Queries rely on exact keywords, proper nouns, or domain terminology the embeddings may not capture |
| Full-text | Keyword search matching exact terms | Queries are phrased differently from the documents or use synonyms and paraphrases |
| Hybrid | Runs ANN and full-text in parallel, merged with Reciprocal Rank Fusion (RRF) | Recommended general-purpose starting point; the most robust strategy for a mixed workload |
| Hybrid + reranker | Hybrid, then a cross-encoder re-scores the results | Latency budget can't absorb the extra time (typically under 1 second per query) |
The evaluation documentation includes a lookup table from result pattern to cause, and that table is the core of this diagnosis. If hybrid scores clearly above ANN, the queries benefit from keyword matching and the cause is the default ann query type, so switch to hybrid in production. If ANN and hybrid score about the same, keywords add nothing and the simpler ANN is fine. If full-text clearly beats ANN, the embeddings are not capturing your domain. The cause is then the embedding model chosen when the index was built, not the query, and the fix is to fine-tune the embedding model or use full-text search. If the reranker lifts the metrics significantly, enable it if your latency budget allows. Wide confidence intervals mean you don't have enough queries yet, so you can't conclude anything until you add more. Finally, if every strategy scores low, no query setting is the cause. The problem is the quality of the data itself.
| Pattern | Root cause indicated | Remediation |
|---|---|---|
| Hybrid significantly better than ANN | Queries benefit from keyword matching; ANN-only query type | Use hybrid search in production |
| Full-text significantly better than ANN | Embeddings may not capture your domain well | Fine-tune the embedding model or use full-text search |
| Reranker improves metrics significantly | Initial ordering is weak; cross-encoder adds lift | Enable reranker if latency budget allows |
| Wide confidence intervals | Not enough queries for reliable comparison | Increase the number of evaluation queries |
| All strategies score low | Data quality or relevance issues | Work through the retrieval quality guide (data preparation) |
Checkpoint 4 of 5· Check yourself
A support agent's evaluation shows full-text well ahead of ANN on a catalogue full of internal part codes. Which configuration is most likely the root cause?
When full-text clearly beats ANN, the documentation reads that as embeddings failing to represent the domain, and suggests fine-tuning the embedding model or switching to full-text.
“Embeddings may not capture your domain well.”Source: docs.databricks.com
Checkpoint 5 of 5· Exam question
An engineering team benchmarks their RAG agent's retrieval using both the default Databricks-hosted `gte-large-en-v1.5` embedding model and a third-party `text-embedding-3-large` model on the same evaluation set. The results show both models produce nearly identical retrieval quality scores. Given this finding, what should the team conclude about their retrieval quality bottleneck?
Correct answer: A — The bottleneck is unlikely to be the embedding model, so effort should focus on other levers such as hybrid search, filtering, reranking, or data preparation
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.If every retrieval strategy scores low, the fix is to switch to a different query type or add the reranker.Why is that wrong?
Uniformly low scores across ANN, hybrid, full-text and reranked runs point to problems with the data, not to the query configuration. Fix the data preparation instead.
Covered in From result pattern to root cause
2.A perfect NDCG means retrieval is giving the agent everything it needs.Why is that wrong?
NDCG only checks that results are in ideal order. It can be 1.00 when most results are irrelevant. DCG@10 measures how much useful content was actually returned.
Covered in The built-in retrieval quality evaluation
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Before optimizing retrieval quality, you must have a reproducible evaluation system.”
↩︎ Measure before you change anything“Optimizing without measurement is guesswork.”
↩︎ Measure before you change anything“Focus on relative improvements as you test different strategies, not absolute scores.”
↩︎ Prediction“RAG agents: Missing key context leads to incorrect answers or hallucinations.”
↩︎ Checkpoint - 2.
“Each strategy is also evaluated with and without the reranker.”
↩︎ The built-in retrieval quality evaluation“DCG@10 provides the most complete picture of retrieval quality for most applications”
↩︎ The built-in retrieval quality evaluation“Consider fine-tuning your embedding model or using full-text search.”
↩︎ From result pattern to root cause“This approach compares strategies side-by-side on the same query set.”
↩︎ Key concept“Data quality or relevance issues.”
↩︎ Exam trap 1“DCG answers the question: "How much useful information did the user actually get?"”
↩︎ Exam trap 2“Score relevance: An LLM judge evaluates every query and retrieved document pair on a 4-point relevance scale.”
↩︎ Checkpoint“a bimodal distribution (many 3s and many 0s) might suggest that a query pattern isn't being retrieved well and needs attention.”
↩︎ Checkpoint“Embeddings may not capture your domain well.”
↩︎ Checkpoint - 3.
“The default query type is ann (approximate nearest neighbor).”
↩︎ From result pattern to root cause