Why Your AI Agent's Token Bill Exploded
Nothing failed. The loop just paid for its own history, twelve times per task.
What you'll learn
- Why an agent loop’s input tokens grow with the square of its step count
- What prompt caching actually charges — reads at 0.1×, writes at 1.25×
- The one line that silently stops the cache being read, with no error
Share this passage
Drawing…
Understand it one step at a time
The short runs these in order in about 51 seconds. Here they are written out — pick any step to jump the short straight to it.
The agent cost $4,260 last month
One customer-support triage agent. Four thousand completed tasks in a month. The invoice landed at six times the original estimate.
Frequently asked questions
- Why does an AI agent's token bill grow faster than the number of steps it takes?
- Most agent loops resend the entire conversation history with every step — step 10 re-sends everything from steps 1 through 9. That makes total input tokens grow roughly with the square of the step count, not linearly with it.
- How does prompt caching change the cost of that resent history?
- A cached read is billed at roughly 0.1× the normal input rate, while writing something new into the cache costs about 1.25×. The saving only materializes if the prefix you're resending is byte-identical each time — one line that changes something early in the prompt can silently break the cache and put you back to paying full price.
Free app · no app store
These are built for a phone
Every short is drawn at full portrait height, the shape a phone already is. Installed, it opens full-bleed with no address bar across the top — and the whole library reads offline.
Read deeper on Azure
Twenty seconds gets the shape of an idea across. These go into how it behaves in production.
-
How Azure Protects a Mobile App: The Full Request Flow, Layer by Layer
Front Door, your API, Microsoft Entra ID, authorization and a private database — the five layers that stand between a mobile app and its data on Azure.
-
Your Mobile App Is Leaking Its API Key — and Obfuscation Will Not Save It
An APK or IPA is a zip file. Anything hardcoded inside it, including your API key, is already public — and here is how to get it off the phone for good.
-
What Is Microsoft Foundry (Azure AI Foundry)? The Services, and When You Actually Need Them
A plain tour of Microsoft Foundry for developers: the resource, projects, models, agents, tools, and when Azure OpenAI on its own is enough.
More shorts
-
Azure
Secure a Mobile API in 5 Steps
8 steps · 25s
-
Azure
What Happens When You Tap Sign In
8 steps · 25s
-
Azure
Your Mobile App Is Leaking Its API Key
8 steps · 26s
Get new posts by email
New technical articles, Azure AI and GitHub Copilot updates, and upcoming events. No spam, unsubscribe anytime.
MSDEVBUILD
/ blog Multi-Stack Dev Build · by Suthahar Azure