Why Keyword Search Fails on Large Product Catalogs

A search that returns nothing when the answer exists

A shopper types "warm waterproof boots." The catalogue has a dozen products that fit — under titles like "insulated rain boot," "thermal wellington," "cold-weather trail boot." Keyword search returns zero results, because no single product title contains all three of those words in that order. This is the pattern behind why keyword search fails on a large product catalogue: it matches the words you typed against the words in the data, and nothing else.

Keyword search matches strings, not meaning

A lexical search engine tokenises the query, maybe stems it ("running" and "run" collapse), and looks for products whose indexed text contains those tokens. Good ones add prefix matching and typo tolerance. None of that gives the engine any notion that "waterproof," "water-resistant," and "GORE-TEX" are related ideas, or that "toddler" and "2T" describe the same shopper. Every one of those connections has to be spelled out by a human, or it doesn't exist.

Why catalogue size makes it worse

At a few hundred SKUs, one person can hold the vocabulary in their head and keep a synonym list current. Scale changes the maths:

• More products, more phrasings. A large catalogue describes the same concept dozens of ways, especially when suppliers write their own copy. "Sofa" / "couch" / "settee" / "2-seater" all live in the data at once.
• Multi-supplier feeds are inconsistent. Titles, attributes, and categories don't follow one style guide. Some are keyword-stuffed, some are terse, some are in another language.
• The long tail of queries grows faster than the catalogue. More products means more specific searches — "boots under £100 for wide feet" — and lexical search has no way to answer an attribute question phrased as a sentence.
• Ranking degenerates. When hundreds of products match "black t-shirt," string relevance can't order them meaningfully, so the first screen is arbitrary.

The synonym-list treadmill

The usual compensation is a merchandising team maintaining spreadsheets of synonyms and manual query redirects, plus a steady stream of "why doesn't product X show up when I search Y" tickets to work through. On the catalogue behind our semantic search project a four-person team was doing exactly this against a feed adding roughly 3,000 SKUs a week. That is not a process that scales — every new supplier and every seasonal range adds vocabulary the list doesn't cover yet.

What actually breaks, concretely

• Vocabulary mismatch — the shopper's word and the catalogue's word for the same thing don't match.
• Descriptive and natural-language queries — "something smart to wear to an outdoor wedding" has no keyword to latch onto.
• Attribute queries — size, fit, material, price band expressed in prose rather than selected as filters.
• Brand-versus-generic — searching a generic term when products are listed by brand name, or vice versa.
• Morphology and plurals beyond what stemming catches, and cross-language supplier copy.

Where keyword search is still the right tool

• Known-item lookups. Part numbers, SKUs, ISBNs, exact model names. The shopper knows precisely what they want and types an identifier — exact string matching is correct here, and semantic similarity would only add noise.
• Small, stable catalogues. If the range is a few hundred items and the vocabulary rarely changes, a maintained synonym list is genuinely enough.
• Faceted navigation. Structured filters (category, size, colour, price) are not free-text search and don't have this problem. Keyword search plus good facets covers a lot of ground.
• Latency or cost floors. Lexical search has almost no per-query cost. If that matters more than recall on vague queries, it's a real trade-off, not an oversight.

FAQ

Isn't fuzzy matching or stemming enough? Those handle typos and word forms — "boot" versus "boots," "recieve" versus "receive." They don't bridge the gap between two different words for the same concept, which is where most zero-result searches come from.

Can we just keep adding synonyms? Up to a point. The cost scales with catalogue growth, it stays reactive (you add the synonym after a shopper fails to find something), and it still can't answer attribute or natural-language queries.

Does semantic search replace keyword search entirely? Usually it's a hybrid — semantic retrieval for descriptive queries, lexical matching kept for exact identifiers. The point is to stop relying on string matching for queries that are about meaning.

ISTRALLEN builds semantic catalogue search that understands what shoppers mean instead of matching strings; see AI for Retail.