How Often Should a Product Catalog Be Re-Indexed for Search?

The question has the wrong shape

"How often should we re-index the catalogue" assumes one clock. Catalogue re-indexing frequency is really three different deadlines for three kinds of change, and lumping them into a single cron job either wastes compute or leaves out-of-stock products ranked on the first screen.

Three clocks, not one

Price and stock: near-real-time. These don't require re-embedding — they're metadata attached to the vector record, not part of the text the embedding is built from. But they have to track the live catalogue closely. A search that ranks a sold-out item first burns trust in the whole feature. Push these updates on change, not on a schedule.

Semantic content: event-driven re-embed. When a title, description, category, or key attribute changes, that product needs re-embedding. Minutes to a couple of hours of lag is usually fine. Trigger on the write, queue the work, and filter so a price change doesn't count as a semantic change. On our semantic search project this is an async trigger-and-queue keyed to semantic-field changes only.

Everything: full re-index. Only when the embedding model changes, the composed-field recipe changes, or the vector schema changes. Rare, deliberate, run once.

Event-driven beats scheduled at scale

A supplier feed that adds thousands of SKUs a week and edits many more exposes the problem with a nightly batch: new products stay invisible to search for up to 24 hours, and the batch re-embeds the entire catalogue to catch the small fraction that actually changed.

A trigger-and-queue approach handles it the other way around: the write fires an event, a worker re-embeds just that product, and the index is current within the time it takes the queue to drain — without touching the millions of rows that didn't change.

A concrete setup

One workable shape, and roughly what runs on our project:

• A write to a product row fires a database trigger.
• The trigger inspects which columns changed. Price or stock only? Update the metadata on the vector record and stop — no embedding call. A semantic field (title, description, category, attributes)? Enqueue a re-embed job.
• A background worker drains the queue, calls the embedding model for each queued product, and writes the new vector.
• The full re-index is a separate, manually-run job, used when the embedding model or the composed-field recipe changes.

The filter at the trigger is what keeps cost sane: a catalogue touched thousands of times a week by price and stock updates would otherwise generate thousands of pointless embedding calls for no change in what the vector represents.

When a scheduled batch is actually fine

• Small catalogue with infrequent edits. A full re-embed is cheap, and simpler than wiring up triggers.
• A feed that lands once a day anyway. Match the batch to the feed; there's nothing to gain from checking more often than the data arrives.
• Content changes that genuinely aren't urgent, with stock handled on a separate fast path.

The failure modes at each end

• Too slow: a stale index. New SKUs missing, renamed products unfindable, and — if stock rides the same slow clock — unavailable items ranked highly.
• Too eager: a synchronous re-embed on every write puts an external API call on the critical path of catalogue updates, so the feed stalls. Re-embedding on price changes spends money for zero relevance gain.

Where this changes

• Re-embed volume outgrows a single queue worker, or a second system needs the same change events — move to change-data-capture tooling instead of one trigger-and-queue.
• Near-real-time semantic freshness is genuinely required — a live editorial catalogue, say — which calls for dedicated streaming rather than a queue.
• Multi-region — keeping the index consistent across regions is its own project, scoped separately.

FAQ

Do price and stock changes need re-embedding? No. They're metadata on the vector record. Re-embed only when the text the embedding is derived from changes.

Is a nightly full re-index enough? For a small, slowly changing catalogue, yes. For a continuous supplier feed, event-driven re-embedding of changed products plus a fast path for stock is the pattern that holds up.

How far behind can the index safely be? Semantic content can lag minutes to hours. Price and stock should be as close to live as you can manage, because ranking an unavailable product is the trust-killer.

ISTRALLEN builds catalogue indexing that keeps pace with a live supplier feed without re-embedding the world nightly; see AI for Retail.