The repeated chat history hiding in my $20 AI budget

A $20 monthly AI budget can cover plenty of useful work, but a long conversation can spend it faster than expected. The surprising part is often not the latest question. It is the earlier material sent alongside that question. To understand context windows, start with a small monthly bill and follow the tokens behind it.

Start with a budget and two sample prices

Imagine you use an AI assistant for coursework, debugging, and reviewing small projects. You set aside $20 for the month. For this walkthrough, assume input costs $1 per million tokens and output costs $5 per million tokens. These are invented rates for easy arithmetic, not prices for a particular model.

Input tokens are the material the model receives. Output tokens are what it generates. Tokens are pieces of text rather than exact words, so use the token counts reported for your requests instead of estimating from page counts.

Read the context window as a working-space limit

A context window is the maximum amount of material a model can handle within one request. Depending on the model and API, that allowance includes input and generated output, with separate limits sometimes applying to output.

Think of it as the size of a desk, not a monthly subscription allowance. A larger desk lets you spread out more notes. It does not make every note free to read, and you are not normally charged for empty space in the window.

Your input can include instructions, your latest question, earlier messages, documents, and tool results. The visible question may therefore be only a small part of the billed input.

Calculate a month of compact requests

Start with 400 requests during the month. Suppose each request averages 2,000 input tokens and 500 output tokens. That produces 800,000 input tokens and 200,000 output tokens.

• The input costs $0.80 at the sample rate.
• The output costs $1.00 at the sample rate.
• The monthly total is $1.80.

That leaves plenty of room in your $20 budget. More importantly, it gives you a baseline. You can now see whether a change in your workflow increases request count, input size, output length, or all three.

Add the history that travels with each question

Now imagine those same 400 requests average 30,000 input tokens because your client includes a growing conversation and lengthy tool results. Keep the average output at 500 tokens.

The monthly input becomes 12 million tokens, costing $12. Output still costs $1, bringing the total to $13. You asked the same number of questions and received equally long answers, but the bill rose by $11.20.

Many chat clients send earlier messages again so the model can continue coherently. The model does not automatically remember a previous API call for free. Some systems manage conversation state differently, so check your client's behavior and the provider's usage accounting rather than assuming every interface works identically.

Check caching before blaming the window size

Repeated input does not always carry the full ordinary input price. Prompt caching can discount eligible repeated material, depending on the model, provider, and request structure. Other charges or token categories may also apply.

Use the $13 result as an uncached illustration, not a prediction. In MyRelay, you can inspect each request's tokens and cost. Compare a short, fresh conversation with a long-running one doing similar work. Look at the billed cost alongside token counts, since caching can make equally sized requests cost different amounts.

Keep useful context and remove stale context

Before your next session, separate what the model needs now from what merely accumulated. Keep relevant requirements and recent findings. Remove duplicate documents, obsolete tool results, and unrelated conversation. For a new task, consider a fresh chat with a concise handoff, while preserving details needed for correctness.

Then review a few requests each week. If you want one personal, pay-as-you-go budget across multiple model providers, create a MyRelay account and use its request-level records to check your assumptions.

A large context window is useful capacity. Your spending habit is deciding when to fill it.