Edge vs Cloud Inference for In-Store Computer Vision

The choice, and why connectivity usually decides it

For in-store computer vision you either run detection on a small device in each store, or upload the shelf photos to a central server and run it there. Edge vs cloud computer vision in retail is often framed as a cost or accuracy question, but for a store network it's usually settled by one fact: what the stores' internet connections can actually move.

The case against cloud-only

Retail store connectivity is frequently business broadband or 4G/LTE failover, not a data-centre uplink. Typical mobile upload speeds run roughly 10-15 Mbps. A shelf photo at a resolution that can resolve individual SKU labels is realistically a few megabytes, so each one takes several seconds just to transfer — before TLS handshake, server queueing, and the return trip. Multiply that by the number of photos in one store audit, across 150-plus stores, and network transfer time becomes the dominant latency in the pipeline. That's the opposite of what a timely stockout alert needs.

The case for edge

• On-device accelerators are fast and cheap enough. A low-power inference accelerator runs a mid-size detector in single-digit milliseconds at roughly a 2-watt budget — small enough to put in every store with no meaningful power or cooling footprint. Published benchmarks show around a 20x speedup over desktop-CPU inference for comparable models, though a specific model needs benchmarking on the actual hardware rather than assuming from an adjacent one.
• Only the result travels. A structured detection event — SKU, confidence, shelf position — is a few bytes, not a few megabytes. On our computer vision project that's what makes the alerting layer viable over the same store connections that ruled out cloud-only inference.
• Privacy. The photo, which incidentally captures staff and shoppers, never leaves the store.

The case for cloud, where it fits

• Reliable high-bandwidth connectivity at every site — fewer, larger stores on a corporate network.
• A model too large for affordable edge hardware, or one that changes so often you don't want to push it to a fleet of devices.
• A retraining loop that genuinely needs the raw images centrally — with the bandwidth and privacy trade-offs handled deliberately.

The hybrid

Run detection on-device for the real-time alert path, and separately compress and batch-upload a deliberate sample of images on a schedule for retraining and spot audits. You get timely alerts without betting the pipeline on the connection, and you still feed the model.

What actually leaves the store

A detection event is a handful of fields — SKU, confidence score, shelf position, a timestamp, a store ID. A few dozen bytes. Compare that to a multi-megabyte photo, per shelf, per audit, per store. That ratio is the whole argument: on-device inference turns a bandwidth problem into a rounding error, and it's why the alerting layer works over the same links that made cloud upload impractical.

The fleet is the hidden cost of edge

Edge trades a bandwidth problem for a fleet-management problem. Every store has a device to provision, monitor for health, and update when the model changes. Across 150-plus sites that's real, ongoing operations work — staged model rollouts, a way to tell which devices are offline, a hardware refresh plan. A portable model format helps by letting one build run across whatever mix of hardware the estate accumulates, so you're not maintaining a separate optimised model per device type on top of everything else.

Where this stops being right

• A small number of well-connected stores. Cloud is simpler — no fleet of edge devices to provision and update.
• A model that genuinely won't fit on affordable edge hardware, and can't be shrunk enough.
• A use case that needs the raw image centrally. Then the store-connectivity problem has to be solved first, not worked around.

FAQ

Does edge mean a GPU in every store? No — a low-power accelerator roughly the size of a USB stick, drawing about 2 watts. Not a server, not a rack.

How do we update the model on 150-plus devices? A managed, staged rollout — one of the real operational costs of edge. A portable model format keeps a single build working across mixed hardware as the fleet grows.

Can we start cloud and move to edge later? You can, but for a large store network the connectivity maths usually forces the edge decision early rather than after a failed cloud pilot.

ISTRALLEN builds in-store computer vision that runs on-device and sends only the detection result; see AI for Retail.