Choosing an Embedding Model for Product Search: Small, Large, or Multilingual

Three choices, one that matters most

Picking an embedding model for product search comes down to three questions: a smaller or larger model from the same family, whether you need a multilingual one, and managed versus self-hosted. The cost difference between them compounds at catalogue scale, so it's worth getting deliberate about.

What "larger" buys you

A few points on retrieval benchmarks. The current large model scores around 64.6% on the MTEB average against roughly 61% for the previous generation, and the small model in the same family trails large by under 3 points. On an English catalogue that's a small quality difference.

What it costs

The large model runs roughly five times the per-token price of the small one. At catalogue scale with continuous re-embedding on a supplier feed, that difference isn't a rounding error — it's a line item that grows with the catalogue. On our semantic search project the small model was the default for exactly this reason: under 3 points of quality difference didn't justify five times the cost at a 120,000-SKU catalogue that re-embeds thousands of products a week.

When you actually need multilingual

Non-English product copy, or shoppers searching in more than one language. Here the gap between model generations is large, not marginal — the current model scores around 54.9% on multilingual retrieval against roughly 31.4% for the previous one. A multi-supplier catalogue with inconsistent, sometimes non-English descriptions is exactly the case where the multilingual number is the one that decides it.

Self-hosted open-source

Top open models score close to the managed ones — roughly 63 against 64-65 on MTEB — so raw quality isn't the deciding factor. The real trade is operational: self-hosting means owning GPU inference and its scaling, monitoring, and failure modes; a managed API means paying per token with no infrastructure to run. Self-hosting starts to pay off only once token volume is high enough that the API cost exceeds the infra and ops cost.

The switching cost

Changing the embedding model means re-embedding the whole catalogue — a one-off bulk job — and a full re-index. So pick deliberately. But it's a one-time migration, not a permanent lock-in: the embeddings are portable and the pipeline doesn't change.

Three worked cases

• An English-only apparel catalogue, 200k SKUs, re-embedding ~5k a week. Small model, managed API, no multilingual. The quality gap to large doesn't justify five times the per-token cost at that re-embed volume.
• A multi-market grocer with German and French product copy. Multilingual model — the retrieval gap between generations on non-English text is too large to ignore — but still the managed API, and the smaller tier if the family offers a multilingual small.
• A 5M-SKU marketplace with a dedicated MLOps team and very high query volume. Now self-hosting is worth evaluating: the per-token API bill at that scale can exceed the cost of running your own GPU inference, and the team exists to operate it.

Run the bake-off, don't trust the leaderboard

Before you commit, take 50 real queries from your logs, embed the catalogue with two candidate models, and have someone judge the top 10 results for each query side by side. Benchmark averages are computed over datasets that aren't your catalogue; a half-day bake-off on your own data and your own queries is worth more than a leaderboard position.

Where this stops being right

• A multilingual model you don't need is paying for capability you won't use — the small English model is cheaper and close in quality.
• Self-hosting below the volume crossover buys you infrastructure to run for no cost saving.
• Benchmark numbers are a starting point — run your own query set against two candidates before committing.

FAQ

Small or large? Small is the default for an English catalogue at scale — under 3 points of quality difference at roughly a fifth of the cost.

When do I need multilingual? Non-English product copy or multilingual shoppers. The multilingual retrieval gap between model generations is large.

Should we self-host? Only when per-token API cost exceeds the GPU and ops cost of running your own. Raw quality is competitive either way.

ISTRALLEN picks the embedding model against your catalogue's language mix and re-embedding volume, not a leaderboard; see AI for Retail.