15,000 GPS pings a second. Do not put that in a queue.

Understand it one step at a time

The short runs these in order in about 32 seconds. Here they are written out — pick any step to jump the short straight to it.

1. 1

   60,000 drivers. Every 4 seconds

   Every car reports where it is. Nobody asked for a reply, and nothing needs a lock.

   MSDevRide · driver telemetry · live

2. 2

   1.3 billion pings a day

   Each one is a few hundred bytes, and each one is worthless four seconds later.

   15,000 events/sec · 1.3B/day

3. 3

   A queue is the wrong tool

   Four teams need the same ping. A queue hands each message to exactly one consumer, then deletes it.

   queue · one consumer wins, three lose

4. 4

   Fix: Azure Event Hubs

   Not a mailbox — an append-only log. Producers append, and nothing is handed out or removed.

   ride-telemetry · 4 partitions

5. 5

   One log, split into partitions

   The partition key hashes to a lane. Every ping from one driver lands in the same lane, in order.

   partition key = driverId · ordering per partition

6. 6

   Reading doesn't delete

   Each consumer group keeps its own offset over the same events. Four teams, one copy of the data.

   3 consumer groups · independent offsets

7. 7

   A slow reader can't hurt the rest

   fraud-scan is 41 seconds behind. The live map is still at the head, and no event was dropped.

   fraud-scan lag 41s · live-map lag 0s

8. 8

   Replay yesterday, no re-ingest

   A new team rewinds to an old offset inside the retention window and reads the same events again.

   analytics · rewound to 06:00 · retention 7 days

9. 9

   Queue vs stream

   One is a job to be done and destroyed. The other is a fact to be read by anyone, twice if needed.

   Service Bus · work — Event Hubs · events

10. 10

    Commands queue. Facts stream

    “Charge this card” belongs in a queue. “Driver moved here” belongs in a log everyone can read.

    Azure Event Hubs · ride-telemetry