MyRelay

← Back to the notebook

I budget for three tries, not one perfect answer

Start with the task you keep asking twice

You ask an AI model to explain a bug. The first answer is short and useful. You send the same prompt again, and it produces a long tutorial that misses the point. A third attempt finally gives you something usable. The expense is not just the price of one answer. It is the total cost of reaching an answer you can use.

To see where that money goes, choose one small task you actually repeat, such as reviewing a function or explaining a lecture concept. Use a task whose answer you can check. This walkthrough works best with a narrow question, not an open-ended request to improve an entire project.

Write down what a usable answer must contain

Before sending the prompt, define success in plain English. For a bug explanation, that might mean identifying the faulty assumption, explaining its effect, and suggesting one correction. Decide whether examples are necessary and how much explanation you need.

Put those requirements into the prompt. Asking for three short paragraphs will not guarantee identical answers, but it gives the model a clearer target than asking it to be helpful. It also makes evaluation less dependent on whether an answer sounds confident or polished.

Run three controlled attempts with a spending ceiling

Set a small amount you are comfortable spending on the experiment. Then make up to three independent requests using the same model, prompt, and available settings. Start each request without previous conversation history. Otherwise, the second attempt is no longer receiving the same input.

Identical visible prompts do not guarantee identical answers. Generation can involve sampling, and service-side behavior can also affect results. Lowering randomness, where supported, can reduce variation, but it is not a promise of exact repetition. Do not spend your test budget chasing perfect reproducibility unless your application genuinely requires it.

Compare the bill alongside the answer

For each attempt, record whether it passed your requirements, its input and output token counts, and its cost. MyRelay lets you see tokens and cost for each request, which makes this comparison easier than estimating expense from answer length alone.

Different answers can consume different numbers of output tokens. Some models may also bill reasoning tokens that are not visible in the final response. Input charges can differ when context or cache treatment changes. Use the recorded charge as your spending measure, rather than assuming that a response twice as long costs exactly twice as much.

Calculate the cost of getting something usable

Add the charges for all attempts, including the disappointing ones. Divide that total by the number of answers that met your requirements. That gives you a rough cost per usable answer for this small test. If none passed, the test has not established a usable cost; it has identified a workflow that needs adjustment.

For example, suppose three attempts cost a hypothetical six cents altogether and two are usable. Your observed cost per usable answer is three cents. That is more informative than pointing to the cheapest individual response. Three attempts are only a starting sample, though, not a reliable forecast for every future task.

Change one expensive habit before switching models

Look for the simplest improvement. If answers ramble, request a tighter scope and use an output limit where available. Leave enough room for a complete answer, since truncation can create another paid retry. If answers misunderstand the task, add the missing constraint rather than repeatedly sending an unchanged prompt.

When an answer is mostly correct, a focused correction may be better than a fresh attempt. However, follow-ups can carry earlier conversation tokens into the next request. Compare their actual cost. Change one thing at a time so you can tell whether the improvement came from the prompt, settings, or model.

Keep a retry rule you can afford

Choose a stopping rule before your next session: one initial attempt, one targeted correction, then manual review, for example. Keep whichever rule produces acceptable results within your budget. To track this habit across your requests, create a MyRelay account and use its request-level costs to check your totals. The aim is not identical prose every time. It is a dependable path to useful work without unlimited paid retries.

More to read