Cheap First, Expensive Second: A Model Cascade

A cheap model can handle much of an application's routine work. A more expensive model can help when a request needs stronger reasoning or a better answer. A cascade connects the two: start with the cheaper option, check the result, and escalate only when there is a concrete reason.

The saving depends on how often the first answer is usable. If almost every request reaches the second model, you pay for two attempts and add waiting time. Build the cascade around one narrow task, then measure whether it earns its place.

Choose a task with a checkable result

Start with something you can evaluate without asking another model to judge every answer. Extracting fields from a message, categorizing support requests, or answering questions from supplied notes can work well. Open-ended writing is harder because a polished answer may still miss what the reader needs.

For a first experiment, imagine extracting an event's name, date, and location from student club announcements. Define what counts as success before choosing models: required fields must be present, dates must be interpretable, and extracted details must come from the announcement.

Assign one model to each attempt

You could try DeepSeek V4 Flash for the first attempt and DeepSeek V4 Pro for escalation. Treat this as a pairing to test against your examples. Check current prices and observed quality before deciding that the price difference makes the extra routing worthwhile.

Keep the task instructions and output requirements consistent across both attempts. With MyRelay, you can access both through one key and one budget, which makes the experiment easier to manage without juggling separate balances.

Make the first attempt easy to verify

Give the first model the announcement, the fields you need, and a clear instruction to leave missing information unknown. Guessing a venue or silently choosing between conflicting dates should count as failure, even if the result looks complete.

Keep the response short enough for the task. An extraction request rarely needs a paragraph explaining every field. Smaller outputs can reduce cost and make checks simpler, though brevity alone does not establish correctness.

Escalate on evidence you can inspect

Run inexpensive checks before accepting the result. Confirm that required fields exist, the date is valid, and quoted names or locations appear in the source. If the announcement contains two plausible event dates, mark the result for escalation.

Do not rely only on the model saying it feels confident. A confident answer can be wrong. Use explicit failures, source conflicts, and task-specific rules as your main signals. Where correctness cannot be checked automatically, review a sample of accepted answers to estimate how many mistakes slip through.

Give the second model the original evidence

When a check fails, send the original announcement and the same requirements to the stronger model. Include the reason for escalation, such as two conflicting dates. If you include the first answer, identify it as an unverified attempt so it does not become the assumed truth.

Apply the checks again. Set a maximum of two model attempts per request for this experiment. If neither produces a supported answer, return an unknown value or ask the user to clarify. A stronger model cannot recover information the source never supplied.

Calculate the cost of both attempts

Average cascade cost equals the average first-attempt cost plus the escalation rate multiplied by the average second-attempt cost. In a hypothetical example, if the first attempt costs one budget unit, the second costs eight, and one request in five escalates, the average is 2.6 units. Sending every request directly to the second model would cost eight units under those assumptions.

Use measured costs because input length, output length, and repeated context affect the result. MyRelay shows tokens and cost for every request, helping you account for both attempts. Also track total response time: escalated requests wait for two calls.

Keep the cascade only if the results justify it

Test a small collection of representative announcements, including ambiguous and incomplete examples. Compare the cascade with sending everything to the stronger model. Measure cost, acceptable answers, missed errors, and waiting time.

Create a MyRelay account to run a small trial with a spending limit you are comfortable with. Expand the cascade only when your results show that cheaper accepted answers preserve the quality your task needs.