I test two examples before paying for ten

A few examples can make an AI response more consistent. They can also become a recurring charge for instructions the model already understands. Before I attach a page of demonstrations to every request, I test whether two examples solve a problem that plain instructions cannot.

Here is a small walkthrough using a common student or solo-developer task: sorting incoming support messages into billing, access, and bug categories.

Start with a clear instruction and no examples

Write the simplest useful prompt first. Explain the three categories, say that the model must choose one, and specify the response format. If messages can belong to several categories, define which category takes priority. Otherwise, examples may end up compensating for an unclear rule.

Collect about twenty realistic messages, including obvious cases and a few ambiguous ones. Decide the expected answers yourself before running the test. Keep this evaluation set separate from any messages you later use as examples.

Run the instruction-only prompt and record correctness, format compliance, input tokens, output tokens, and cost. This is your baseline. Without it, a longer prompt can feel more reliable without actually being better.

Find the mistake that an example could explain

Read the failures before adding demonstrations. If the model ignores a missing category definition, improve the instruction. If it invents account details, examples will not give it access to the missing facts.

Few-shot examples are most useful when the desired behavior is easier to show than describe. That includes a particular writing style, a subtle classification boundary, or an unusual but consistent output convention.

Suppose the model labels every failed payment as an access problem because the customer cannot sign in afterward. An example can show that the underlying payment issue belongs in billing. That demonstrates a decision boundary, rather than simply repeating an easy case.

Add two short demonstrations with different jobs

Choose one straightforward example that demonstrates the required response format and one boundary case that demonstrates the troublesome distinction. Give each a sample input and the correct response, clearly separated from the live message.

Keep both examples compact. Long customer histories, greetings, and explanations consume tokens unless they teach something necessary. Make sure the examples agree with your written rules and use invented details rather than private customer information.

Then rerun the same evaluation set with the same model and settings. You are testing the examples, so avoid changing the model, instructions, and output limits at the same time.

Compare the extra spending with the errors avoided

Count both semantic mistakes and formatting failures. A correct category wrapped in unwanted prose can still break a downstream workflow. For a small test, inspect individual results instead of trusting a single percentage.

Next, calculate the added input volume. If your demonstrations add 300 tokens per request, 1,000 requests add 300,000 input tokens before any caching benefit. Multiply that volume by the applicable input price. Include changes in output length and retries when comparing total cost.

MyRelay lets you inspect each request’s tokens and cost, which helps check whether the longer prompt actually costs what you expected. Create a MyRelay account if you want to track this experiment within one pay-as-you-go budget. Do not assume repeated examples are discounted; check the applicable caching rules and recorded usage.

Remove examples before deciding to add more

If two examples improve results, remove one and test again. You may discover that only the boundary case matters. If neither helps, return to the shorter prompt rather than adding demonstrations by habit.

Add another example only when a recurring failure needs a distinct lesson. Ten nearly identical examples usually provide less useful coverage than two carefully chosen ones. For hard format requirements, also consider supported structured-output features and application-side validation instead of relying on examples alone.

Keep the smallest prompt that survives fresh messages

Finally, test the winning prompt on new messages. Repeatedly tuning against the same twenty cases can make a prompt look stronger than it is.

Keep the smallest version that meets your accuracy and reliability needs. Few-shot prompting earns its place when the cost of extra input is outweighed by fewer mistakes, retries, or manual corrections. When a clear instruction already works, leaving examples out is not cutting a corner. It is avoiding a charge that buys you nothing.