How a Shelf-Monitoring Computer Vision Pipeline Is Architected

The requirement that becomes an architecture problem

Catch stockouts and shelf-placement errors close to when they happen, across a whole store network, without a data centre's worth of hardware or bandwidth at every location. That constraint rules out more of the obvious shelf-monitoring computer vision architecture than any accuracy number does: it decides where inference runs, how results travel, and what a "new store" costs to add.

Capture

A fixed camera or a staff device produces shelf photos. Consistency of angle and lighting is worth engineering for; it makes every downstream stage easier and the model more accurate.

Detection model

A single-shot detector: a YOLOv8-class model, a mid-size variant. It gives enough accuracy for shelf-level product and gap detection while fitting the inference budget of edge hardware. Two-stage detectors (Faster R-CNN) were ruled out on speed; transformer detectors (DETR-style) on training cost and immature support on edge accelerators. The largest model variants are built for server GPUs, not a device in an aisle.

Inference runtime

A portable format, ONNX Runtime, with multiple execution providers, so one trained model runs across whatever edge hardware ends up installed instead of maintaining a separate optimised build per device type. It also keeps the rollout from being locked to a single accelerator vendor as the fleet grows.

Where inference runs

On-device at each store. Only a small structured detection event (SKU, confidence, shelf position) leaves the store; the raw photo never travels. On our computer vision project this was a connectivity decision first, but it's also what makes everything after it work over constrained store links.

Transport

A lightweight publish-subscribe protocol: MQTT. Each store publishes detection events, a central broker receives them, and the dashboard subscribes once to get events from every store. A 2-byte message header against roughly a kilobyte of HTTP headers matters when 150-plus stores each publish small, frequent events, and the broker handles retained messages and delivery to a store on a flaky connection without a bespoke integration per site.

Routing

Confidence-based. Auto-alert above a high-confidence threshold; route the ambiguous middle band to a lightweight human review queue before it becomes a task on a store associate's list. This is what keeps store teams acting on the feed instead of tuning it out: alert fatigue is the failure mode a single fixed threshold walks into.

Delivery and the feedback loop

Alerts land in the store-ops dashboard. Every "wrong alert" a reviewer corrects becomes labelled data for the next retraining pass, so accuracy improves instead of plateauing.

Adding a store

The payoff of the portable runtime and the pub-sub transport is what it takes to bring a new location online: install the device, provision it onto the broker with the store's ID, point it at the camera feed. No new integration, no per-store model build, no bespoke API endpoint to stand up and secure. A new store is configuration on top of an existing pipeline, which is the only way a rollout reaches 150-plus sites on a fixed timeline.

Updating the model across the fleet

Because detection runs on-device, a model improvement has to reach every device. A staged rollout (a few canary stores, then a region, then the rest, with a way to roll back) is part of the architecture, not an afterthought. The portable format keeps this to one artefact regardless of the hardware mix underneath.

Where this stops being right

• A handful of well-connected stores. A simpler cloud pipeline may be enough, no edge fleet to manage.
• A model that won't fit edge hardware and can't be shrunk.
• A need for depth-of-stock, not front-facing availability. That's shelf sensors or RFID; a photo sees the front row.
• No capacity to staff the review queue that confidence routing depends on.

FAQ

Why not upload photos to a cloud model? Store connectivity can't reliably move multi-megabyte images at the volume and timeliness a stockout alert needs. On-device inference sends bytes, not megabytes.

Why MQTT instead of a REST API per store? Many stores to one dashboard, small frequent messages, flaky links: that's exactly the shape publish-subscribe handles. A REST endpoint per store re-implements retained messages and delivery guarantees the broker already provides.

How accurate does the model need to be? Accurate enough that confidence routing keeps the human-facing hit rate high. Design around routing, not a single headline accuracy number.

ISTRALLEN builds shelf-monitoring pipelines end to end: on-device detection, portable runtime, pub-sub transport, confidence routing; see AI for Retail.