Giving Merchandising Control of Search Relevance Without Engineering Tickets

Relevance is never "done"

Seasonal ranges come and go, promotions run for a week, lines get discontinued, a new supplier names things oddly. The merchandising team needs to nudge search results constantly. If every nudge is an engineering ticket, relevance drifts on a sprint cadence while the catalogue moves daily. Merchandising control of search relevance is the practice of giving that team the controls directly.

What merchandisers actually need to do

• Boost — push a product or category up for a query or a query pattern, for a season or a promotion.
• Bury — push down out-of-stock, discontinued, low-margin, or off-brand results.
• Pin — fix a specific product at position one for a specific query.
• Filter or redirect — a query always returns a curated set, or sends the shopper to a category landing page.
• Synonyms and spelling — add "couch equals sofa" without a deploy. A good semantic model covers most of this; the console handles the residue a supplier feed throws up.

The design that makes "no redeploy" true

Relevance rules are stored as data the team edits through a console and read at query time — not code shipped through a release. On our semantic search project that console is a small internal React app used by a four-person, non-technical merchandising team; the rules are rows applied on top of the retrieve-and-rank pipeline, so a change takes effect immediately without an engineer in the loop.

Guardrails so it doesn't turn into a mess

• An owner and an expiry on every rule. A Black Friday boost that never turns itself off is a bug waiting to surface in February.
• An audit log — who changed what, when, and why.
• A "why did I get these results" view that shows which rules fired for a given query.
• Before/after on conversion for significant changes, so a well-intentioned rule that hurts gets caught.

How a rule gets made

A merchandiser searches "winter coats" ahead of the season and sees last year's clearance stock ranking above the new range. In the console: create a boost rule scoped to that query and the current-season coat category, set it to expire when the season ends, save. The next search picks it up — no ticket, no deploy, no waiting for a release. When the season's over the rule expires on its own instead of quietly distorting results into spring.

Rules as a diagnostic

The set of rules is a map of where the underlying system is weak. A cluster of boosts around one supplier's products usually means that supplier's titles and attributes are poor and the content needs fixing. A pile of buries for out-of-stock items means the index isn't tracking stock closely enough. Reviewing the rule list every few weeks — not just adding to it — turns it into a to-do list for the model and the catalogue content.

Where the model ends and the console begins

The embedding model and the re-ranker should carry the bulk of relevance. The console is for business overrides the model can't know: this promotion, this margin priority, this line being wound down. If merchandisers are hand-fixing hundreds of ordinary everyday queries, that's a signal the underlying relevance or the catalogue content needs work — not a reason to add more rules.

Where this stops being right

• A tiny catalogue with a stable range. A short synonym list and occasional engineering help is enough; a console is overhead.
• No merchandising function. If nobody will own and use the console, don't build it — the rules will go stale unattended.
• A team that would over-rule the model into incoherence. Start with a few rule types and tight guardrails, and widen only as the team shows it can manage them.

FAQ

Won't merchandisers just break relevance? With owners, expiries, an audit log, and a before/after check on big changes, no — and the alternative is relevance frozen between deploys while the catalogue keeps moving.

Does this replace the ranking model? No. It's a layer of business overrides on top. The model and re-ranker do the heavy lifting; the console handles what they can't know.

How many rules is too many? When rules are compensating for everyday queries the model should be getting right on its own. That's a prompt to fix the model or the content, not to keep adding rules.

ISTRALLEN builds search with a merchandising console so relevance is tuned by the team that owns it, not through the release cycle; see AI for Retail.