Common Mistakes When Indexing a Catalog for Semantic Search

The relevance problems start at the index

The model and the vector store get most of the attention, but the errors that actually tank search relevance are almost all in what you put into the index and how you keep it current. These are the semantic search catalogue indexing mistakes that come up again and again, and the ones worth designing against from the start.

1. Embedding the wrong text

Dumping the raw product description — HTML tags, shipping and returns boilerplate, keyword-stuffed marketing copy — into the embedding dilutes the signal. The vector ends up representing "generic product page" as much as the product. Embed a clean composed field instead: title, key structured attributes, category, and a concise description. Not the rendered page.

2. One giant blob instead of structured fields

Concatenating everything into a single string flattens what matters. The title and a few salient attributes should carry more weight than the twelfth line of the care instructions. Keep structured attributes — size, colour, material — as filterable fields alongside the vector, not melted into it, so a shopper can still narrow results precisely.

3. Indexing content but ignoring price and stock

A vector index that's out of sync with the live catalogue will confidently rank a product that sold out an hour ago in the top slot. That single experience damages trust in the whole search feature. Price and stock don't need re-embedding — they're metadata on the vector record — but they have to track the catalogue closely. On our semantic search project keeping embeddings in the same database as the catalogue is what made "results always reflect current price and stock" enforceable rather than aspirational.

4. Re-embedding synchronously on every write

Supplier feeds touch product rows thousands of times a week. If every write triggers a synchronous re-embed, you've put an external API call on the critical path of catalogue updates — feeds slow down or stall. Use an async trigger-and-queue: the write fires an event, a worker re-embeds in the background. And filter it to semantic field changes, so a price tick doesn't queue a pointless re-embed.

5. No re-index path at all

The opposite failure: embed the catalogue once at launch, then never again. The thousands of new SKUs the feed adds each week are invisible to search, and renamed products stop being findable under their new names. The re-embed pipeline isn't an enhancement — without it the index rots from day one.

6. No evaluation set

Shipping relevance changes with no way to tell whether they helped is how search quality drifts sideways for months. Keep a fixed set of real queries with their expected top results, and run it before and after every change. It doesn't need to be elaborate — a few dozen queries that matter catches most regressions.

The multi-supplier wrinkle

Catalogues fed by many suppliers have inconsistent, sometimes non-English product copy. A stronger multilingual embedding model absorbs a lot of that. The residue — a supplier's invented category name, an odd abbreviation — is best handled by a boost-and-bury console the merchandising team controls directly, so fixing a bad ranking doesn't need an engineering ticket.

Where not to over-engineer

• Small, stable catalogue: a nightly full re-embed is simpler than triggers and perfectly adequate.
• No non-English copy: don't pay for a multilingual model you won't use; a smaller English model is cheaper and close in quality at catalogue scale.
• Low query volume: skip personalization and heavy A/B infrastructure. A fixed evaluation set is enough to tune against.

FAQ

What text should actually go into the embedding? A clean composed field — title, key attributes, category, short description. Not raw HTML, not boilerplate, not keyword-stuffed copy.

How fresh does the index need to be? Fresh enough that price and stock are right — that's non-negotiable. Semantic content can lag by minutes to hours without anyone noticing.

Do I need a dedicated vector database to do this well? Not at catalogue sizes in the low hundreds of thousands. Keeping vectors in the database you already run puts embeddings and catalogue data in one system, which is exactly what makes freshness tractable.

ISTRALLEN builds catalogue indexing pipelines that stay in sync with a live supplier feed; see AI for Retail.