How a Shelf-Monitoring Computer Vision Pipeline Is Architected

The requirement that becomes an architecture problem

Catch stockouts and shelf-placement errors close to when they happen, across a whole store network, without a data centre's worth of hardware or bandwidth at every location. That constraint rules out more of the obvious shelf-monitoring computer vision architecture than any accuracy number does — it decides where inference runs, how results travel, and what a "new store" costs to add.

Capture

A fixed camera or a staff device produces shelf photos. Consistency of angle and lighting is worth engineering for; it makes every downstream stage easier and the model more accurate.

Detection model

A single-shot detector — a YOLOv8-class model, a mid-size variant. It gives enough accuracy for shelf-level product and gap detection while fitting the inference budget of edge hardware. Two-stage detectors (Faster R-CNN) were ruled out on speed; transformer detectors (DETR-style) on training cost and immature support on edge accelerators. The largest model variants are built for server GPUs, not a device in an aisle.

Inference runtime

A portable format — ONNX Runtime — with multiple execution providers, so one trained model runs across whatever edge hardware ends up installed instead of maintaining a separate optimised build per device type. It also keeps the rollout from being locked to a single accelerator vendor as the fleet grows.

Where inference runs

On-device at each store. Only a small structured detection event — SKU, confidence, shelf position — leaves the store; the raw photo never travels. On our computer vision project this was a connectivity decision first, but it's also what makes everything after it work over constrained store links.

Transport

A lightweight publish-subscribe protocol — MQTT. Each store publishes detection events, a central broker receives them, and the dashboard subscribes once to get events from every store. A 2-byte message header against roughly a kilobyte of HTTP headers matters when 150-plus stores each publish small, frequent events, and the broker handles retained messages and delivery to a store on a flaky connection without a bespoke integration per site.

Routing

Confidence-based. Auto-alert above a high-confidence threshold; route the ambiguous middle band to a lightweight human review queue before it becomes a task on a store associate's list. This is what keeps store teams acting on the feed instead of tuning it out — alert fatigue is the failure mode a single fixed threshold walks into.

Delivery and the feedback loop

Alerts land in the store-ops dashboard. Every "wrong alert" a reviewer corrects becomes labelled data for the next retraining pass, so accuracy improves instead of plateauing.

Adding a store

The payoff of the portable runtime and the pub-sub transport is what it takes to bring a new location online: install the device, provision it onto the broker with the store's ID, point it at the camera feed. No new integration, no per-store model build, no bespoke API endpoint to stand up and secure. A new store is configuration on top of an existing pipeline, which is the only way a rollout reaches 150-plus sites on a fixed timeline.

Updating the model across the fleet

Because detection runs on-device, a model improvement has to reach every device. A staged rollout — a few canary stores, then a region, then the rest, with a way to roll back — is part of the architecture, not an afterthought. The portable format keeps this to one artefact regardless of the hardware mix underneath.

Where this stops being right

• A handful of well-connected stores. A simpler cloud pipeline may be enough — no edge fleet to manage.
• A model that won't fit edge hardware and can't be shrunk.
• A need for depth-of-stock, not front-facing availability. That's shelf sensors or RFID; a photo sees the front row.
• No capacity to staff the review queue that confidence routing depends on.

FAQ

Why not upload photos to a cloud model? Store connectivity can't reliably move multi-megabyte images at the volume and timeliness a stockout alert needs. On-device inference sends bytes, not megabytes.

Why MQTT instead of a REST API per store? Many stores to one dashboard, small frequent messages, flaky links — that's exactly the shape publish-subscribe handles. A REST endpoint per store re-implements retained messages and delivery guarantees the broker already provides.

How accurate does the model need to be? Accurate enough that confidence routing keeps the human-facing hit rate high. Design around routing, not a single headline accuracy number.

ISTRALLEN builds shelf-monitoring pipelines end to end — on-device detection, portable runtime, pub-sub transport, confidence routing; see AI for Retail.