Search Relevance & Faceted Discovery

When a higher _score still does not prove a better learner result

A learner searches for debug flaky async tests. Five catalog items match. A language facet removes two. Among the three remaining results, a high-scoring introductory article ranks above a lower-scoring advanced debugging guide. That ordering can be perfectly understandable to the search engine and still be weak for an experienced learner's information need.

This lesson builds Search Relevance & Facet Evaluation Contract v1. The goal is not to declare one ranking algorithm universally best. The goal is to make visible what came from query scoring, what was removed by binary filtering, how facets shaped the candidate set, what explicit relevance judgments say, and what the selected metric actually measures.

Lesson boundary and learning routes

MKN-007 owns Search Relevance & Faceted Discovery. MKN-003 is the hard conceptual prerequisite. The previous (MKN-006) and next (MKN-008) lesson outcomes remain separate; this lesson does not absorb credential architecture or documentation monitoring and does not mutate a live search index.

Foundation starts with the sixteen-field map and one baseline. Repair isolates the first broken evidence dimension, such as missing judgments or taxonomy drift. Experienced learners must defend the separation among score, filter, facet and judgment without turning vendor defaults or a metric into hidden authority.

Mental model: intent → retrieval → score + filter → rank → facet → judgment → metric → keep/tune/fallback

Decision flow:

learner intent → candidate retrieval → scored query clauses + binary filters → ranked candidate set → facet navigation → relevance judgments → rank evaluation → keep / tune / fallback / hold

A query score represents how well a document matches under the engine's scoring model. A filter includes or excludes candidates without producing relevance scores. A facet supports navigation or candidate-set restriction under a taxonomy. A judgment is explicit evidence about suitability for a specific information need. A metric summarizes ranking quality against those judgments. Collapsing these layers creates false confidence.

Foundation contract: sixteen auditable dimensions

query_id and learner_intent identify the problem. candidate_set records retrieved items. relevance_score preserves scoring evidence and filter_state records binary inclusion/exclusion. rank_position captures the final ordering of displayable candidates.

facet_dimension, facet_value, and taxonomy_version preserve navigational context. judgment_set and relevance_grade bind relevance labels to the specific intent. evaluation_metric and metric_result summarize ranking quality against those labels. zero_result_state and fallback_route preserve discovery failure explicitly, while review_owner makes accountability visible.

A complete row is evidence, not automatic permission to say the ranking is good.

Deep explanation: keep FACT, INFERENCE, and DESIGN separate

FACT — query versus filter. Current Elasticsearch documentation defines query context as calculating relevance scores in addition to matching, while filter context answers a binary match question without score calculation. Moving a scored clause into filter context therefore changes ranking semantics, not merely execution mechanics.

FACT — ranking evaluation. Elasticsearch _rank_eval accepts typical search requests plus rated documents and can calculate metrics such as precision, mean reciprocal rank, and discounted cumulative gain. Metric availability does not select the right metric for a learning goal.

INFERENCE — audit facets separately. If a result disappears, a reviewer should be able to tell whether its score was too low or whether a filter/facet removed it before ranking. Facet counts are not quality proof by themselves.

DESIGN — MAKNOW. Query fixtures, intent, judgment policy, relevance grades, metric, threshold, zero-result fallback, taxonomy version, and owner are local design authority. Vendor documentation does not supply that authority.

Worked examples: debugging, invalid shortcut, and transfer

Baseline — production debugging. Three English results remain after filtering. Engine order is intro-async → advanced-debugging → checklist, while grades for the production-debugging intent are 1,3,2. NDCG@3 ≈ 0.7364. The synthetic local fixture defines a design threshold of 0.85, so the bounded state is RANK_TUNING_REVIEW_REQUIRED. This does not say Elasticsearch is wrong; it says the ranking does not yet meet the local bar for these judgments.

Invalid shortcut — scores remain, judgments are removed. Without explicit relevance judgments, a valid relevance metric against the learner's information need cannot be defended. The state is HOLD_RELEVANCE_JUDGMENTS_MISSING; engine score is not its own relevance label.

Transfer — beginner explanation. The same content set and query text can serve a different information need. For a beginner, grades may become 3,1,2, making the same order NDCG@3 ≈ 0.9721. That result is valid only when judgments were rebuilt for the new intent; copying baseline labels is forbidden.

Failure modes that must fail closed

If query identity or intent is unknown, the candidate set cannot be reconstructed, a scored clause was converted to a filter without recomputation, required score evidence is unknown, filter/rank state is inconsistent, judgments or grades are missing, the metric is unsupported, the recorded metric disagrees with the visible ranks/grades, taxonomy version is unknown, a facet value no longer maps, the owner is missing, or transfer reuses baseline labels, the decision remains HOLD.

Zero results are not “zero relevance.” An empty candidate set requires an explicit zero_result_state and fallback_route. A fallback can broaden a query, recover spelling, relax a facet, or route to navigation, but it must remain a separate decision rather than silently showing unrelated results.

Alternatives and trade-offs: score, metric, facet, and threshold

Precision@k is simple but ignores the exact positions of relevant items within top-k. MRR is useful where the first relevant answer matters strongly. DCG/NDCG incorporates graded relevance and rank position. Metric selection should follow the task rather than API convenience.

More facets can improve exploration but a weak taxonomy can create false precision. Hard filters control the candidate set but may remove highly relevant results before scoring. A hard threshold makes QA explicit, but without intent-specific judgments it becomes a decorative number. The Contract keeps these trade-offs separate.

Experienced bridge: relevance is not a fixed property of a document

Relevance is a relationship among document, query, learner intent, and context. Introductory material can be excellent for a beginner and weak for production debugging. That is why judgment fixtures and facet taxonomies need versioned identity.

For ranking experiments, retain an immutable query fixture reference, version judgments with intent, record metric configuration, and trace facet/filter mutations independently. If AI proposes relevance labels, keep them as candidate evidence; they are not canonical ground truth until governed authority and a review owner accept them.

Guided and independent practice

1. Run the baseline and reproduce NDCG@3 from rank + grades. 2. Remove judgments and explain why score cannot replace them. 3. Convert one scored query clause into a filter and require recomputation. 4. Version the facet taxonomy and leave one old value unmapped; taxonomy drift must remain visible. 5. Make the candidate set empty and record a defensible fallback route. 6. Change intent to beginner and rebuild judgments without looking at the baseline labels.

For independent practice, remove the expected state. The learner gets only the sixteen Contract dimensions, visible rules, and synthetic fixture.

Verification evidence: what a reviewer must reproduce

A reviewer must reconstruct query/intent, candidate set, scoring evidence, filter inclusion/exclusion, final rank, facet/taxonomy, judgment grades, metric configuration/result, zero-result/fallback, and owner from visible artifacts. The reviewer must also be able to recompute the recorded metric and distinguish a ranking change caused by scoring from a candidate-set change caused by filtering.

The deterministic lab covers the baseline, a strong rank, missing judgments, query/filter collapse, score/filter/rank defects, metric mismatch, taxonomy drift, zero-result fallback, missing owner, changed intent without judgment rebuild, and forbidden baseline-oracle reuse.

Assessment and transfer: one query text, two information needs

Assessment has four blocking parts: explain the MKN-007 boundary and one non-goal; build the baseline Contract; diagnose the “scores without judgments” shortcut; and recompute the beginner transfer from scratch.

Transfer passes only when learner_intent changes, new relevance grades are recorded, the metric is recomputed, and facet/filter use is reviewed against the new intent. Keeping the same query text and content set gives no permission to reuse baseline relevance labels.

References, freshness, and authority boundary

Official Elastic documentation for Query and filter context and Ranking evaluation was rechecked on 2026-09-13. Claims about query scoring/filter semantics and _rank_eval remain FACT and should be rechecked by 2027-01-15 or earlier on material platform change.

Facet-audit separation, relevance-judgment policy, metric/threshold choice, taxonomy versioning, fallback, and HOLD/REVIEW_REQUIRED states are MAKNOW DESIGN authority. The lesson keeps vendor facts separate from product decisions.