MyRelay

← Back to the notebook

I check three AI claims before buying another answer

A polished AI answer can turn a nonexistent function into an afternoon of debugging. It can also give a student a convincing quotation that never appeared in the assigned reading. The expensive part is often not the request itself. It is the time spent building on something false.

My habit is to check three important claims before asking for another answer. This walkthrough turns that habit into a repeatable process, using a common task: asking an assistant to explain a library and suggest an implementation.

Start by deciding what would be costly to get wrong

Before sending your request, identify the parts that need to be accurate. For a library task, those might be the supported version, the function signature, and the behavior when an operation fails. For an essay, they might be a quotation, a publication date, and the evidence behind the central argument.

Choose three claims whose failure would change your next action. This keeps verification manageable. You do not need to investigate every sentence equally, but a quick check is not enough for consequential medical, legal, financial, or security decisions.

Ask for checkable claims rather than general confidence

Give the assistant the relevant documentation or reading excerpt when you have it. Ask it to distinguish statements supported by that material from assumptions and suggestions. Request precise references, such as a documentation section or a page in the supplied edition.

You might ask it to identify the three claims your implementation depends on and explain what evidence would confirm each one. This makes the answer easier to inspect. It does not make those claims true: models can invent references and explanations as readily as they invent facts.

Avoid treating a confidence percentage as evidence. A confident sentence, even one accompanied by a number, still needs an external check.

Check the first claim against the actual source

Suppose the assistant recommends a function and says it accepts a particular option. Open the official documentation for the version you use. Find the function, read its parameters, and check any version notes. A search result summary is a lead, not the final evidence.

For academic work, locate the actual paper or book passage. Confirm that the author, title, and publication details match, then read enough surrounding text to establish whether the source supports the claim. A real citation can still be attached to an unsupported conclusion.

If you cannot locate the source, mark the claim as unverified. Do not let the assistant replace a missing reference with another plausible-looking reference indefinitely.

Test the behavior with the smallest safe example

Next, test the claim that can be observed directly. For a coding task, isolate the suggested behavior in a disposable environment with no production credentials or sensitive data. Check one normal input and one failure case before integrating the suggestion.

A successful run establishes only what you tested. It does not prove compatibility with every version or show that the approach is secure. Still, a small test often catches invented interfaces and misunderstood return values faster than another round of discussion.

For noncoding work, use the equivalent direct check: recalculate the arithmetic, compare a quotation with the original, or trace a summary back to the supplied passage.

Use a second answer only to target unresolved doubts

When a claim remains unclear, ask a narrow follow-up containing the conflicting evidence. Request a revised answer grounded in that evidence, rather than asking the assistant to confirm its earlier response.

A second model can suggest overlooked checks, but agreement between models is not independent verification. Both may repeat the same common error. Prefer documentation, original sources, and reproducible observations over a vote between assistants.

MyRelay makes the spending side visible through each request's token usage and cost. Use that visibility to notice when repeated retries are replacing actual investigation. If you want one pay-as-you-go budget for this workflow, create a MyRelay account. Keep follow-ups focused instead of resending an entire conversation.

Save the evidence beside the decision

Before using the result, record which claims you verified, the source or test that supported them, and anything still unresolved. Then remove unsupported details from the final work or clearly state the uncertainty.

The stopping point is not when the assistant sounds reassuring. It is when the evidence is strong enough for the decision you need to make. Three deliberate checks often buy more reliability than three more answers.

More to read