This is the depth of analysis behind every project we build. We publish it as confirmation of how we actually work — the real options we weighed, the numbers that ruled them out, the architecture that survived contact with the client’s actual catalog — so our process stays as transparent to clients as the results are. This project’s version of "carving from stone" wasn’t about picking a model — it was about deciding what NOT to build ourselves, because the tempting failure mode on a search project is reinventing infrastructure that already exists.
The client ran a mid-market retail storefront with a catalog fed by multiple suppliers — roughly 120,000 SKUs, growing by a few thousand a week as new suppliers onboarded. Their search was SQL ILIKE pattern matching against product titles. It returned poor matches for anything but exact keyword hits — a shopper searching "warm waterproof boots" got nothing if no product title literally contained all three words, even when a dozen matching products existed under different phrasing.
A four-person merchandising team was compensating manually: maintaining spreadsheets of manual search synonyms and constantly firefighting "why doesn’t X show up when I search Y." That’s not a scalable process at 3,000 new SKUs a week.
The ask was: make search actually understand what shoppers mean, keep it fast as the catalog keeps growing, and give merchandising a way to tune relevance without filing an engineering ticket every time. That last point — a non-technical team needing control without redeploys — shaped the frontend decision as much as any backend choice.
Why we didn’t just buy Algolia: Algolia’s own published pricing bills on two dimensions — search requests and records — starting at $0.50/1k additional requests and $0.40/1k additional records past the free tier, and their AI-feature tier (NeuralSearch, personalization) jumps to $1.75/1k requests, a 3.5x increase over standard search pricing for the exact capability this project needed. At 120k SKUs with weekly catalog churn and meaningful query volume, that’s a forecastable but real ongoing cost that scales with the business rather than flattening out. This isn’t an anti-Algolia argument in general — for a smaller catalog or a team that wants zero search infrastructure ownership, it’s usually the right call. It’s a fit argument for this catalog’s size and growth trajectory specifically.
Why "generative recommendations," not just "ranked results," ruled out an off-the-shelf retriever alone: neither Algolia nor Elasticsearch generates grounded natural-language recommendations natively — both are retrieval engines that return ranked structured results. Algolia’s own material positions generative answers as a separate bolt-on layer ("Agent Studio") that still has to call an external LLM; Elastic’s documentation frames the same pattern as "wire us into your RAG pipeline." The real differentiator isn’t whether Algolia can be connected to an LLM — it’s that owning the retrieval→rerank→generation chain ourselves means full control over prompt engineering, cost per generation call, and ranking logic, instead of paying for a black-box version of the same idea through someone else’s AI-feature pricing tier.
Why not Elasticsearch either: its vector search is capable, but it’s a second system to run, patch, and scale independent of Postgres — for a catalog this size, that’s operational overhead bought for capabilities pgvector already covers at this scale (see §2 below). We’d revisit this the moment query volume or vector count outgrows what a single Postgres instance handles comfortably.
A custom FastAPI retrieval service. Not because managed search is bad — because at 120k SKUs with a 4-week window to also ship reranking and a merchandising console, owning the pipeline was cheaper in total cost of ownership and gave us the generative layer the client actually asked for, not just better-ranked keyword results.
Why pgvector held up at this scale: Supabase’s published HNSW benchmark (September 2023) tested pgvector on OpenAI-dimensioned (1536-d) embeddings from ~224k up to 1 million vectors, and found HNSW indexing delivering roughly 3x the queries-per-second of the older IVFFlat index at equal or better recall on that hardware tier. Our catalog — 120k SKUs today, growing toward maybe 300-400k over the next couple of years — sits comfortably inside the range that benchmark actually tested, not past it.
Why keeping vectors in the same Postgres instance mattered more than raw search speed: the real risk here wasn’t retrieval latency, it was staleness — a vector index that’s out of sync with the live catalog will confidently rank a product that sold out an hour ago. With embeddings and catalog rows in the same database, price/stock changes and semantic re-embedding can be reasoned about as one system instead of two systems that can silently drift apart. We use an async trigger-and-queue pattern rather than a synchronous re-embed on every write, which would otherwise block catalog updates on an external API call — a real risk on a table touched thousands of times a week by supplier feeds.
The detail that saved real engineering time: because price and stock changes fire the same write trigger as a genuine content change (title, description, category), we filter at the trigger level — only re-embed when the semantic fields change, not on every price tick. Full CDC tooling like Debezium exists for teams that need decoupled, replayable event streams across multiple downstream consumers, but a single-instance Postgres setup with filtered triggers covers this catalog’s actual requirement without a second piece of infrastructure to operate.
Postgres with pgvector, synced via an async trigger-and-queue pattern filtered to semantic-field changes only. We’re tracking two triggers to revisit this: catalog size approaching the multi-million-vector range, or a genuine need for CDC-level replayable event streams across more than one consuming system.
Why we picked OpenAI’s current-generation embedding model over the previous generation, specifically: OpenAI’s own January 2024 announcement gives real, dated numbers — text-embedding-3-large scores 64.6% on the MTEB benchmark average versus 61.0% for the previous ada-002 model, a modest but real gain on English-language retrieval. The more decisive number for a supplier-fed catalog with mixed-language product data is MIRACL (multilingual retrieval): text-embedding-3-large jumps to 54.9% versus ada-002’s 31.4% — a genuinely large improvement, not a marginal one. Since this catalog pulls from multiple suppliers with inconsistent (and sometimes non-English) product copy, the multilingual number is the one that actually mattered for our decision.
Why not self-hosted open-source embeddings: on the current MTEB leaderboard, top open models (e.g. BGE-M3) score close enough to OpenAI’s and Cohere’s models (roughly 63 vs. 64-65) that raw quality isn’t the deciding factor — it’s genuinely competitive. The real trade-off is operational: self-hosting means owning GPU inference capacity and its scaling, monitoring, and failure modes; a managed embedding API means paying per token with zero infrastructure to run. For an 8-week build window with no dedicated MLOps function on the client side, that operational trade-off — not model quality — is what ruled out self-hosting here.
text-embedding-3-small, not -large — the MTEB gap between small and large is under 3 points, and small is roughly a fifth of large’s per-token cost. At catalog scale with continuous re-embedding on supplier updates, that cost difference compounds; the quality difference at our scale doesn’t justify paying it.
Why vector similarity alone wasn’t enough: a bi-encoder ranks by how close two embeddings are in vector space — fast, but it’s an approximation of relevance, not a direct judgment of it. Reranking benchmarks across independent, cross-dataset comparisons consistently show cross-encoder reranking improving nDCG@10 by roughly 5 to 15 points over bi-encoder-only retrieval — a real, reproducible-enough range we treat as the defensible baseline expectation, distinct from Cohere’s own marketing claim of "up to 25% better on challenging retrieval tasks," which we’re citing as a vendor best-case ceiling, not an average to promise a client.
Why the added latency was worth it here, specifically: reranking means a second network round-trip scoring the top 50-200 vector-search candidates before returning the final top 10 — real added latency, typically tens to a couple hundred milliseconds. For a low-stakes internal search tool, that overhead often isn’t worth it. For e-commerce catalog search where result relevance directly drives conversion, we made the opposite call: retrieve broad-but-imprecise cheaply with vector search, then spend the extra latency narrowing to a precise top 10 — because a shopper who doesn’t find the right product in the first screen of results doesn’t scroll, they leave.
Cohere Rerank on the top-N vector search candidates before serving results. This is the layer most directly responsible for the published conversion lift — retrieval finds plausible candidates, reranking is what makes the top results actually the right ones.
One label ("search"), two different frontend problems. A widget built on a full framework runtime is the wrong shape for a third-party embed regardless of which framework it is — every storefront-facing widget we ship is a small, predictable, framework-agnostic bundle for exactly this reason.
React earns its place on the merchandising console for the same reason it does on every internal ops tool we build: mature patterns for data-dense dashboards, no bundle-size constraint, and the team most likely to eventually maintain or extend this tool hires from the deepest available talent pool. Making "no redeploy" genuinely true meant exposing relevance rules as data the merchandising team edits directly, not code that ships through a sprint cycle.
A compiled, framework-agnostic Web Component for the storefront search widget + React for the internal merchandising relevance console.
None of the three published numbers is a coincidence of these five decisions; each is downstream of a specific pair of them:
Published, named case studies from vendors in this space (Algolia’s ManoMano case: +20% conversion; Lacoste: +37%; Gémo: 2.3x) put our client’s +25% squarely inside a credible, unremarkable range for "switched to AI-powered search" outcomes — we’re not claiming an outlier, we’re explaining why landing mid-range required all five decisions above, not just "add a vector database."
Worth stating plainly, because no architecture is permanent:
None of these are failures of the original decision — they’re the conditions under which the same reasoning process, run again, would produce a different answer.
We’re publishing this confidence table on purpose. A client is better served by "here’s what we’re sure of and here’s what we’d re-verify" than by a document that reads clean because the uncertainty got quietly edited out.