This is the depth of analysis behind every project we build. We publish it as confirmation of how we actually work — the real options we weighed, the numbers that ruled them out, the architecture that survived contact with 150+ real stores on real internet connections — so our process stays as transparent to clients as the results are. This project’s version of "carving from stone" wasn’t about picking the most accurate model in a lab benchmark — it was about admitting that a store’s shelf, camera, and internet connection are messier than any benchmark dataset, and building for that mess from the start.
The client ran a multi-store retail chain where product availability was checked by staff physically walking the aisles with a clipboard — a manual audit that happened on a schedule, not continuously, so a stockout that appeared right after an audit could sit unnoticed for hours or days before the next one caught it. Every hour a popular item sat missing from a shelf was a lost sale that a competitor down the street was happy to pick up.
The ask was not "put a camera in every store." It was: catch stockouts and shelf-placement errors close to when they happen, across the whole store network, without requiring a data-center’s worth of hardware or bandwidth at every location. That last constraint — real stores have real, inconsistent internet connections and no server room — ruled out more of the obvious architecture than any accuracy number did.
The second constraint that shaped this as much as the first: store-ops staff already have a full workload. An alert system they learn to distrust — because it’s frequently wrong — gets ignored within weeks, which means the system has to be right often enough, and honest about its uncertainty the rest of the time, to actually get acted on.
Why two-stage detectors were the first thing ruled out: an independent comparative study on detector throughput found YOLOv3 running at 73 FPS against Faster R-CNN’s 12 FPS — better than a 6x gap, driven by the fundamental architectural difference between one forward pass and a two-stage propose-then-classify pipeline. A more recent secondary benchmark comparing YOLOv8’s GPU latency (1.3ms) to Faster R-CNN’s (54ms) found a roughly 40x gap — we’re citing that second figure from a non-peer-reviewed engineering comparison, not betting the whole decision on its exact multiplier, but it’s directionally consistent with the same architectural reason. For 150+ stores generating a continuous stream of shelf photos, the detector needs to keep pace with volume, not just be accurate in isolation.
Why a transformer-based detector wasn’t the fit, despite being the newer research direction: the original DETR paper (Carion et al., ECCV 2020) reports accuracy and runtime "on par with" Faster R-CNN — meaning DETR wasn’t competing in YOLO’s speed category to begin with, it was matched against the two-stage detector already ruled out above. DETR’s training convergence is also a real practical cost: follow-up work (Deformable DETR) cites roughly 500 training epochs against 12-36 for Faster R-CNN. Newer transformer detectors like RT-DETR (2023-2024) do claim YOLO-competitive speed, but on GPU servers — a different deployment target than store-level edge hardware. Our own read here, not a claim from any published source, is that transformer attention operations don’t yet have the same mature, optimized support on edge accelerators that convolutional architectures do — which matters directly once §3 below is decided.
Why YOLOv8 specifically, not an earlier YOLO generation: Ultralytics’ own published benchmarks (COCO val, 640px) show YOLOv8s at 44.9 mAP against YOLOv5s at 37.4 mAP at a comparable model size — a real accuracy gain at the same speed class, from the vendor’s own documentation. We didn’t need the largest variant (YOLOv8x, 53.9 mAP, meant for server-class GPUs) — a mid-size variant gave us the accuracy needed for shelf-level product and gap detection while staying inside the inference budget the edge hardware chosen in §3 could actually sustain.
YOLOv8, a mid-size variant, chosen as a single-shot detector for edge-viable inference speed at accuracy sufficient for shelf image analysis — not the largest or most research-forward option, the one that fit the deployment target.
Why native framework serving didn’t fit a 150+-store rollout: shipping PyTorch’s own runtime to every store edge device means shipping the training framework’s full dependency footprint to hardware that was never chosen with that framework’s performance profile in mind, and it implicitly locks the deployment to whatever hardware that runtime happens to run fastest on.
Why not TensorRT-only: this is a genuinely fast option on NVIDIA hardware specifically — Ultralytics’ own numbers show YOLOv8x running in single-digit milliseconds on an A100 with TensorRT. But a real multi-store rollout doesn’t guarantee every location gets identical NVIDIA-based edge hardware, and locking the whole pipeline to one vendor’s accelerator forecloses the option chosen in §3 (Google’s Edge TPU) unless two separate inference paths are maintained in parallel — real ongoing engineering cost for a store network that’s still growing.
Why ONNX Runtime, specifically: it’s built around exactly this problem — a single exported model format with multiple execution providers underneath — CPU, CUDA, TensorRT, CoreML, NNAPI, and Edge-TPU-compatible paths are all officially documented — letting the same exported model run across heterogeneous store hardware without a separate optimization pipeline per device type. On raw speedup, Microsoft’s own claim of "an average 2x performance gain on CPU" for their production services (Bing, Office, Azure AI) is a real vendor figure, but it’s an average across their specific workloads, not a guarantee for ours. A more granular third-party benchmark we found put a specific PyTorch-vs-ONNX-Runtime comparison at roughly 1.58x, and ONNX Runtime’s own blog cites up to 2.88x with fp16 quantization on a different (NLP) workload. The honest summary: the speedup is real and consistently reported in a 1.5x-3x range depending on model, hardware, and precision — not a single number we’d promise a client without benchmarking their specific model first.
ONNX Runtime as the shared inference layer across store hardware — export the trained YOLOv8 model once rather than maintaining a separate optimized build per device type as the store network grows.
Why cloud-only inference didn’t fit store connectivity reality: retail store internet connections are frequently business-grade broadband or 4G/LTE failover, not data-center-grade uplinks. Typical 4G/LTE upload speeds run roughly 10-15 Mbps per recent mobile network reports (Opensignal, RootMetrics), and a single shelf photo at a resolution that can actually resolve individual SKU labels — realistically a few megabytes — takes multiple seconds just to transfer at that speed, before TLS handshake overhead, server queueing, and the return trip are even counted. We’re flagging this as our own back-of-envelope estimate from public network-speed data, not a published benchmark of this exact scenario, because we couldn’t find one. Multiply that by however many photos a single store audit captures, across 150+ stores, and network transfer time alone becomes the dominant latency in the pipeline — the opposite of what a timely stockout alert needs.
Why on-device Edge TPU inference was the fit: Google’s own published Edge TPU benchmarks show a roughly 20x speedup over desktop CPU inference for comparable models (MobileNet v1: 53ms on desktop CPU vs. 2.4ms on Edge TPU), and the hardware itself draws a genuinely small power budget — roughly 2 watts at its rated 4 TOPS, per Coral’s own datasheet — cheap enough to deploy at every store without a meaningful power or cooling footprint. We’re flagging an honest gap here: Google’s official Coral benchmarks are published for MobileNet, SSD, and EfficientNet-EdgeTPU family models specifically, not for YOLOv8 — there’s no public, vendor-confirmed YOLOv8-on-Coral number to cite, and our own model’s on-device latency was something to benchmark directly during the build rather than assume from an adjacent model family’s published numbers.
Why this wasn’t purely a latency argument: on-device inference also means only the detection result — a small structured payload of SKU, confidence, and shelf position — needs to leave the store over the network, not the raw photo. That’s the detail that made the alerting layer in §4 viable over the same constrained store connections that ruled out cloud-only inference to begin with.
On-device inference on Google Coral Edge TPU hardware at each store, sending only structured detection results over the network, never the raw image.
Why HTTP/REST wasn’t the natural fit for this specific traffic shape: MQTT is an OASIS-standardized protocol (MQTT v5.0, OASIS Standard, March 2019) whose own stated design goal is being "lightweight... suited for use in constrained networks and multi-platform environments" — a description that matches store edge hardware on variable connectivity closely enough to take seriously as more than a buzzword fit. The concrete difference: MQTT’s fixed header is 2 bytes, against a typical HTTP request/response exchange running 800-1,000+ bytes of headers for a payload that might itself be a handful of bytes — figures we’re citing from IoT-industry engineering write-ups rather than a single peer-reviewed source, though multiple independent sources land in the same order of magnitude. For 150+ stores each publishing small, frequent detection events, that per-message overhead compounds in a way it wouldn’t for a handful of large, infrequent uploads.
Why the many-to-one pattern mattered specifically: MQTT’s publish-subscribe model lets every store publish independently to topics without needing to know about or connect directly to the central dashboard, and the dashboard subscribes once to receive from all of them — a documented, standard MQTT architecture pattern, not a custom integration per store. The alternative (each store’s edge device calling a central HTTP endpoint) works too, but re-implements a chunk of what a pub/sub broker already handles by default: retained messages, delivery guarantees for a store on a flaky connection, and one connection point to scale rather than 150+ individual inbound endpoints to secure and monitor.
MQTT, with each store’s edge device publishing structured detection events to a central broker, and the store-ops dashboard subscribing to receive alerts across the full store network.
Why a single fixed threshold, alone, wasn’t enough: shelf photography in a real store is messy — partial occlusion, glare, unusual angles, seasonal or promotional packaging the model wasn’t trained on. A detector confident enough to be useful on clear cases will inevitably also be confidently wrong often enough on the messy ones that a single threshold either lets through enough false alerts to erode store-ops trust in the system, or gets set conservatively enough to avoid that and misses real stockouts instead. The fix isn’t a better threshold — it’s giving uncertainty somewhere to go besides a binary alert/no-alert call.
Why full human review of every detection defeats the point: the entire business case is replacing hours-to-days-late manual audits with faster detection. Routing every single detection through a human reviewer before any action reintroduces a human bottleneck sized to the full detection volume across 150+ stores, not just the genuinely ambiguous fraction of it.
Why confidence-based routing, specifically: sending only the detections in an ambiguous confidence band to a lightweight review queue keeps the fast, automated path handling the large majority of clearly-confident detections at full speed, while the cases most likely to be wrong — and therefore most likely to damage store-ops’ trust in the alerts if wrong — get a quick human check before they become an action item on a store associate’s list. Alert fatigue is a well-documented failure mode across monitoring systems generally: once staff learn that alerts are frequently wrong, they start ignoring all of them, including the correct ones, at which point the system has failed regardless of its underlying detection accuracy. We’re not citing a study specific to retail shelf-monitoring alert fatigue here — we couldn’t find one — this is our own operational reasoning, the same logic that applies to any alerting system a human has to act on repeatedly.
Confidence-based routing — auto-alert above a high-confidence threshold, route the ambiguous middle band to a lightweight human review queue before it becomes a store-ops action item.
None of the three published numbers is a coincidence of the decisions above; each is downstream of a specific combination of them:
We’re publishing these comparisons the same way across every number in this document: as a plausibility check against what the industry reports, not a claim that our client’s results were benchmarked against a specific competitor.
Worth stating plainly, because no architecture is permanent:
None of these are failures of the original decision — they’re the conditions under which the same reasoning process, run again, would produce a different answer.
We’re publishing this confidence table on purpose. A client is better served by knowing which numbers came straight from a primary source — and which ones, like the shelf-photo network latency estimate, are our own reasoning from public data rather than a cited benchmark — than by a document that reads clean because the uncertainty got quietly edited out.