Glossary
Inference economics
What decides the bill for running models: tokens in and out, which model handles each request, and caching. On Anthropic's API, reading a cached prompt costs a tenth of the normal input price or less, depending on the model, and writing it to the 5-minute cache costs 1.25 times.
Cost is a security issue too. OWASP lists unbounded consumption, including denial of wallet, among the top risks for LLM applications: attackers run up the bill or exhaust capacity, so set rate limits, quotas, timeouts and budgets. Caching has its own risk: a 2025 study found seven API providers, OpenAI among them, sharing prompt caches across users, which lets timing reveal what others asked. Anthropic keeps caches separate between organizations.