How Cache Reads Are Billed and Why They Are Cheap

If you reuse a long prompt, you may notice that some input tokens cost less on later requests. Those tokens can appear as cache reads. Understanding this charge helps you estimate the cost of coding sessions, study assistants, and other tools that repeatedly send the same background material.

Walk through a simple example: an assistant receives a fixed set of instructions and reference notes, followed by a new question each time. The repeated material is where caching can help.

Start by separating repeated context from the new question

Suppose your request contains 10,000 tokens of instructions and notes, plus a 500-token question. Without a cache hit, all 10,500 tokens are processed under the applicable input pricing. The answer generates output tokens, which have their own price.

On the next request, you send the same instructions and notes with a different question. A supported caching system may reuse processing associated with the repeated beginning of the prompt. The new question still requires fresh processing, and the answer still incurs output charges.

Follow the first request as it prepares reusable context

The first eligible request establishes the context that later requests may reuse. How this happens depends on the provider and model. Some systems cache eligible prompts automatically. Others require explicit cache settings or designated boundaries.

Pricing also varies. A provider may charge a separate cache creation rate, sometimes higher than ordinary input pricing, or handle cache creation within its standard input charges. Minimum prompt lengths and retention periods can apply. Check the pricing for your selected model before assuming that the first request will save money.

Watch the next request divide into two input categories

When the next request qualifies for a cache hit, its input can split into cached tokens and fresh tokens. In our example, assume all 10,000 repeated tokens qualify as cache reads while the 500-token question remains fresh input.

The input charge then combines two calculations: cached tokens multiplied by the cache read rate, and fresh tokens multiplied by the ordinary input rate. Divide token counts by one million when prices are quoted per million tokens. Add output charges and any separately billed cache creation or storage charges to calculate the full request cost.

Use sample prices to see what the discount changes

For illustration only, assume ordinary input costs $2 per million tokens and cache reads cost $0.20 per million tokens. These are invented rates, not a MyRelay price quote.

Processing all 10,500 input tokens at the ordinary rate would cost $0.021. With a qualifying cache hit, the 10,000 cached tokens cost $0.002, and the 500 fresh tokens cost $0.001. The input total becomes $0.003, a saving of $0.018 for that request.

This example excludes output and any cache creation or storage charges. A long answer could still account for most of the final bill, even when the input receives a substantial discount.

Understand what the lower read price pays for

Cache reads are cheap because the provider can reuse internal computation for matching context instead of repeating all the work needed to process it from scratch. Reading and using that stored state still consumes resources, so the price is generally reduced rather than zero.

The cache does not supply a previously written answer. The model generates a response to your current question using the available context. This is why cached input can be discounted while newly generated output remains billable.

Keep the reusable beginning stable and inspect the bill

Place stable instructions and reference material before changing questions when your application allows it. In systems that match prompt prefixes, changing an early section can prevent later material from qualifying. Expiration, insufficient prompt length, or provider routing can also affect reuse. Repeating text alone does not guarantee a cache hit.

MyRelay gives you one key and one budget across supported providers, with per-request token and cost visibility. Use those records to compare similar requests, accounting for different answer lengths and checking cache usage details where available.

Create a MyRelay account to track your requests and spending as you test repeated prompts. Begin with a small workload, measure the full cost across several requests, and expand only when the savings are visible.