The queue was in order. Your workers were not.

Understand it one step at a time

The short runs these in order in about 50 seconds. Here they are written out — pick any step to jump the short straight to it.

1. 1

   Three events, one cart, in order

   Add an item, remove it, check out. Published in that sequence, one second apart.

   cart-4471 · 3 events · published in order

2. 2

   They were charged for the item they removed

   Checkout ran before the removal. The basket still had the headphones in it when the total was taken.

   cart-4471 · charged $120 · expected $0

3. 3

   The queue was never out of order

   Service Bus delivered these exactly as they were enqueued. Look further down the pipe.

   orders queue · FIFO delivery · no fault

4. 4

   You have sixteen workers

   maxConcurrentCalls defaults to 16. Sixteen threads pulled three ordered messages and raced them.

   host.json · maxConcurrentCalls 16 · default

5. 5

   One worker fixes it, and kills throughput

   Set concurrency to 1 and the order is perfect. So is the queue depth, climbing all afternoon.

   maxConcurrentCalls 1 · ordered · and far too slow

6. 6

   SessionId = cartId

   A session is held by one receiver at a time. In order inside a cart, and still parallel across carts.

   sessions on · one lock per cart · order kept

7. 7

   Ordered with respect to what?

   This is the real decision. Sessions do not turn ordering on — they choose what it applies to.

   sessionid · the partition key you are choosing

8. 8

   The setting that gives the race back

   MaxConcurrentCallsPerSession defaults to 1. Raise it for throughput and you undo everything.

   maxConcurrentCallsPerSession 1 → 4 · order gone again

9. 9

   Turn the other one instead

   Total concurrency is sessions multiplied by calls per session. Scale the sessions, never the calls.

   maxConcurrentSessions 8 × callsPerSession 1 = 8

10. 10

    You cannot add sessions later

    RequiresSession is fixed when the queue is created. Wanting it afterwards means a new queue.

    requiresSession · immutable · create a new entity

11. 11

    And your delayed retry reorders too

    Re-enqueue a copy on a backoff and it lands at the back of its session. Ordering and retry pull against each other.

    scheduled retry · back of the session · order lost

12. 12

    Ordering is a partition key

    Not a checkbox, and not the broker’s fault. It is a decision about what must not overtake what.

    azure service bus · sessions · by design