Verification Checklist

  • ✓Check whether the fixed role setup and long background text in your system prompt sit at the very front of the full prompt, with the current user input placed last
  • ✓Look at your billing detail or console for a field that separates cache-hit from cache-miss token usage, and confirm you are actually seeing that discount applied
  • ✓Confirm which field name corresponds to cache-hit tokens for the specific model and API protocol you call (OpenAI-compatible vs. a vendor's native SDK) — the field name differs across protocols, and checking the wrong field tells you nothing
  • ✓If your vendor offers explicit caching (a mode you must actively declare), confirm you actually added the required parameter per its docs, rather than assuming caching works automatically without any configuration
  • ✓Do not budget costs based on any specific discount ratio or cache lifetime quoted in this article or any third-party source — open the vendor's official pricing docs and check the numbers in effect on the day you place your order

1. The symptom: the same system prompt, but some calls cost less than others

Developers who build on DeepSeek, Qwen, or similar domestic Chinese model APIs often notice something: a call with the same character count in the prompt sometimes costs noticeably less than expected on the bill. The reason usually isn't a sudden price cut — it's that this particular request hit "context caching." If the beginning of this request exactly matches content cached from an earlier call, the repeated portion doesn't need to be recomputed from scratch and is billed at a lower rate; if there's no overlap, or the overlap isn't recognized by the system, the request is billed in full at the standard rate. Many developers don't realize this mechanism exists in the first place, so they never design their prompt structure around it — effectively leaving savings on the table without knowing it.

2. How it works: matching repeated prefixes, billing the hit portion at a lower rate

The core logic of context caching is: for every incoming request, the system checks whether the beginning of that request matches content already retained from a prior call's cache. If it matches, that repeated portion doesn't have to go through a full recomputation — it can be served from cache, so it's billed at a lower per-token rate, one clearly below the standard input-token price. Any portion that doesn't match, or is genuinely new content, is still billed at the standard rate. This typically works automatically with no extra switch required, though some vendors also offer an "explicitly declared cache" mode, where a developer adds a specific parameter to the request in exchange for a more deterministic hit rate. Whether a given vendor's caching is automatic or requires explicit declaration, what conditions determine a hit, and how long cached content is retained all differ by implementation — always check the official API documentation for the specific vendor and model you're calling.

3. Prompt structure: fixed content first, variable content last

This caching mechanism only works if the "prefix" can be matched exactly — most implementations compare content starting from the very beginning of the prompt, and once a difference appears at some point, content further along may fail to count toward this call's continuous prefix match even if it happens to repeat elsewhere. That leads to one very concrete engineering recommendation: put everything that stays the same across calls — system role setup, long background documents, few-shot examples — at the very front of the prompt, and make sure this portion is byte-for-byte identical between calls; put everything that changes on every call — the current user question, a timestamp, a random session ID — at the very end. If you reverse this order, or insert even a single character of variable content in between, the portion that should have repeated may no longer be recognized as the same prefix, and your hit rate can drop noticeably — even if the total character count that repeats looks identical on paper.

4. Where to check cache-hit vs. cache-miss token counts on your bill

The field name that reports cache-hit token counts is not standardized across vendors or API protocols, so you can't assume the same field name applies to every call you make. In practice, start by checking the official documentation for the specific protocol you use (OpenAI-compatible, or a vendor's own native SDK protocol) to confirm the exact field name in the response body, and whether it's counted as part of the total input token count or reported separately. Beyond the field returned per call, most platforms' consoles or billing detail pages also provide a way to filter usage by "cache hit" category, letting you check the overall hit ratio by day or by month without manually summing fields from your own logs. Before scaling up, it's worth running a small batch of real requests first to confirm you actually understand which part of the response is a cache hit and which is a miss, and estimate total cost from that — rather than assuming the ideal hit rate described in the docs matches the real hit rate in your own workload.

5. In fairness: exact hit rules, discount ratios, and cache lifetimes are per vendor's official docs

It's worth stressing that this article only covers what stays true regardless of vendor — that this mechanism exists, roughly how it works, how to structure your prompt around it, and where to check your bill. The exact hit-determination conditions (such as the minimum cacheable token length, or the granularity of prefix matching), the discount percentage applied on a hit, how long cached content is retained, and whether an extra parameter is required — these numbers differ significantly between vendors, and any single vendor may adjust its pricing and rules over time. Vendors also implement "implicit automatic caching" and "explicit declared caching" differently — some offer both with different discount levels, others offer only one. This article deliberately avoids stating a specific discount percentage or cache lifetime, because numbers like that go stale quickly and, once written into an article, can easily mislead readers after a vendor updates its pricing. The correct approach is always to open the specific vendor's official pricing documentation on the day you're actually calling their API and check the rules in effect right then.

6. Bottom line: structure determines your hit rate, the discount itself is per official docs

Context caching is, at its core, a billing optimization mechanism built on "you shouldn't have to pay full price for repeated content twice." Whether it actually saves you money depends heavily on whether your prompt structure follows the "fixed content first, variable content last" principle — not on whether you've done anything more complicated. Putting that structural principle into practice, and periodically checking your bill and console for the real hit ratio, is the concrete gain you can bank on today. As for exactly how much you'll save, and exactly how granular the hit rules are — those numbers change over time, and should always come from the specific vendor's official documentation on the day you check.