Question 1

Are these real API calls or cached results?

Accepted Answer

Real API calls, always. We send your prompt live to each model's API simultaneously. Results are never cached or pre-computed. This means costs, speeds, and token counts you see are exactly what you'd experience integrating the model directly in your app.

Question 2

How accurate are cost projections?

Accepted Answer

The per-run cost is exact - calculated from actual tokens consumed multiplied by each provider's published pricing. The scale projection on 1M runs multiplies cost by 1,000,000. In production, costs vary based on prompt and response length variance, but it's the most accurate estimate available without running production traffic.

Question 3

How does the free daily test works?

Accepted Answer

While you're on the free tier, you get 1 test per day. It resets at 00:01 UTC once spent. You can test up to 5 models per run. Once you top up credits for the first time, all tests run from your credit balance - the daily free test doesn't return if your balance runs out.

Question 4

How workflow tests work?

Accepted Answer

You build a multi-step chain where each step is a separate test. When you add new step, the output from winning model is automatically passed as input for step 2. This mirrors how real AI agent chains work, letting you find the cheapest model for each step without losing quality.

Question 5

What happens to my prompts?

Accepted Answer

Your prompts are stored in your test history so you can revisit results. We never train on or share your prompts. We use anonymised task type and results to improve community model rankings - toggle off anytime. You can delete any test at any time.

Question 6

Why not just use each provider's playground?

Accepted Answer

Setting up all providers individually requires creating 9 accounts, adding billing to each (minimum $90+ in API credits across providers), and spending 24+ hours on integration before seeing any results. We handle all of that - your first result in 30 seconds.

Question 7

When does PRO pays for itself vs Pay as you go?

Accepted Answer

If you're topping up more than $38/month on PAYG credits, Pro costs less overall. At $60/month spend, Pro gives you 37% more tests than PAYG for the same money. Below $38/month, stick with PAYG - there's no point paying the $19 subscription fee.

Question 8

Are you affiliated with any model providers?

Accepted Answer

No. We have no financial relationship with OpenAI, Anthropic, Google, or any other model provider. We don't receive payments for featuring any model or influencing results. Our independence is intentional - a comparison tool that favours one provider is useless. We pay market rates for API access like everyone else.

Pay only for what you use

FREE

Pay as you go

PRO

How markup works

Common questions about model testing

Start cutting cost per inference