Quality check
Write the rule. Other models judge every answer, LLM as a judge.
- 10k free credits
- No card
Describe what a good answer looks like in plain words. Two cheap judge models mark every answer pass or fail, and a third breaks a tie. Then see the cheapest model that passed.

How it works in three steps
From your prompt to the cheapest model that passes your rule.
Run a test
Paste your prompt and run it on the models you pick. The quality check works on a finished test.
Write your pass rule
Say what a good answer is in your own words, for example "answers in under 50 words and doesn't mention competitors."
Read the verdicts
Each answer comes back marked pass, fail or uncertain, and the cheapest model that passed is shown first.
What you get after a check
Cheapest that passes
The answers that passed are ranked by cost, so the cheapest passing model is the first thing you see.
Try cheaper models
Run the same prompt on a few cheaper models you have not tried and judge them with the same rule. The cost is shown first.
Your own rule
You write what counts as a pass. The rule is remembered on the test, so checking again only bills the judging.
Judges from another company
A judge is never from the same provider as the answer it judges.
Uncertain is never a pass
No majority between the judges means the answer is marked uncertain. It is a check, not a guarantee.
The details before you run it
Never the company that wrote the answer. You do not pick the judges.
What it costs
Before a check starts, your balance has to cover every judge call that could happen. Afterwards you are charged once, for what the judging cost. Most answers only need the two cheaper judges.
Checking again
Your rule is remembered on the test. Edit it and check again any time: that bills only the judging, never the original answers.
Where your prompt goes
Your prompt, your pass rule and each answer go to two or three judge models from Anthropic, xAI and Google, companies based in the United States. You do not pick these, and a judge never comes from the company that wrote the answer.
What it does not cover yet
A reliability check's repeats are separate tests, so one quality check on the original does not cover them.
Questions about the quality check
Anything else, write to us or book a demo.
What is LLM as a judge?
Using AI models to grade AI answers. In the quality check you write the rule in plain words; two cheap judge models mark each answer pass or fail, and a third breaks a tie. When the judges cannot agree, the answer is marked uncertain, never passed.
Do I need code to evaluate LLM answers?
No. You write the rule in plain words in the app and run the check on a finished test. No code, SDK or API keys.
What does the quality check do?
You write a pass rule in plain words, for example "Replies with only one word: billing, bug or question." Two cheap judge models read each answer and mark it pass or fail. If they disagree, a third decides. If there is still no majority, the answer is marked uncertain, never a pass. A judge never comes from the same company as the answer it judges. It is a check against your rule, not a guarantee, so look closely at the answers that matter.
What happens when the judges disagree?
A third judge is brought in automatically and the majority decides. If there is still no majority, the answer is marked uncertain instead of being forced into a pass or a fail.
Can a judge favor its own company's model?
No. A model is never used to judge an answer from its own provider.
Does checking again re-run my test?
No. Checking again never re-runs the test or re-bills the answers, only the judging.
Is a pass a guarantee?
No. It is a check against your rule, not a guarantee, so look closely at the answers that matter.
Find the cheaper model before your next invoice
- 10k free credits
- No card
- No API keys