Smart Shopping Cart — action recognition at the edge, live in Qatar
Video action recognition running on a Raspberry Pi bolted to a supermarket trolley, where the enemy was never accuracy. It was latency.
- Real-time accuracy
- 95%
- End-to-end inference
- <80ms
- Action classes
- 6
- Live deployment
- Qatar
Problem
Smart carts need to understand what shoppers are doing in real time — picking up items, placing them back, weighing produce — and they have to do it in a live supermarket, not a lab.
Constraint
The hardest part was never accuracy. It was latency, on constrained hardware. Any delay between a shopper picking up an item and seeing it appear on the display destroys the experience, and the compute was a Raspberry Pi module inside the cart.
Decision
I chose MoViNet — a mobile video network built specifically for temporal understanding on edge devices — over a heavier, more accurate backbone, and then bought the accuracy back through data rather than parameters: 8,000+ action clips collected in-store rather than pulled from a public dataset.
Outcome
95% real-time accuracy across six action classes at sub-80ms end-to-end inference. Deployed and operational at Al-Meera Supermarkets in Qatar, with real shoppers.
Smart carts need to understand what shoppers are doing in real time — picking up items, placing them back, weighing produce. This system brought video action recognition to constrained edge hardware deployed in a live supermarket environment.
Technical stack
The model backbone was MoViNet — a mobile video network designed specifically for efficient temporal video understanding on edge devices. Getting it to 95% accuracy on the action classes we cared about (pick, place, weigh, idle) required significant fine-tuning on a custom dataset collected in-store.
- MoViNet-A2 fine-tuned on 8,000+ action clips across 6 categories
- MQTT pub/sub for real-time event streaming from cart to backend
- Dockerised model serving on Raspberry Pi compute modules in the cart
- Nginx reverse proxy for fleet management across multiple cart units
- MLflow for experiment tracking and model registry
The edge constraint
The hardest part was not accuracy — it was latency. The cart needed to respond in near real-time to user actions. A delay between picking up an item and seeing it appear on the display destroys the UX. We achieved sub-80ms inference end-to-end on the constrained hardware.
Deployed and operational at Al-Meera Supermarkets, Qatar — a live production environment with real shoppers.