Anonymous totals from the accounts that share their results: the winner people pick for each kind of task, and how models stack up on cost, speed and win rate.

The leaderboard in the app, which you see after signing up.
How it works in three steps
From one winner pick to totals across accounts.
Run a test and mark a winner
Picking a winner never re-runs or re-bills anything. It just marks the result you preferred.
The totals are counted
Winners and results from the accounts that share their results are added up anonymously, by kind of task.
See what wins
In the app, Benchmarks shows winner totals by task, with the reasons people gave, and the leaderboard adds Score, win rate, cost and speed.
What you get from the totals
Best model by task
The model picked as the winner most often for each kind of task. After a new comparison, a small model labels the kind of task, when it can tell.
The leaderboard, in the app
After you sign up, the leaderboard ranks models by Score, with tests, win rate, latency and cost. A model gets a Score once it has 10 results.
Why people picked it
Benchmarks in the app shows the reasons people gave for a winner, such as best quality, best value or fastest, and filters by task.
The details before you read the numbers
No prompts, answers, names or account details are shown.
Anonymous totals
Only totals are shown. No prompts, answers, names or organization details.
Your choice to share
Your tests count by default. Turn it off any time in Settings, Privacy ("Share my test results for community benchmarks"), and your past results leave the totals.
Small numbers early on
While there is little data, a total can rest on only a few tests and mostly reflect one account's choices, so read it with the counts beside it.
A Score needs 10 results
A model's Score shows only once it has 10 results, so one lucky win cannot top the table.
Questions about Benchmarks
Anything else, write to us or book a demo.
How is this different from public leaderboards?
Public leaderboards score models on fixed test sets. These numbers come from real tests people run in the app on their own prompts, as anonymous totals from the accounts that share their results.
Where do the numbers come from?
From the tests of accounts that share their results: the winners people pick, and the cost and speed of each result, added up anonymously by kind of task.
Can I keep my tests out?
Yes. Turn off "Share my test results for community benchmarks" in Settings, Privacy. Your past results leave the totals too.
Is my prompt shown?
No. Exact prompts are never shown publicly, and neither are answers, names or account details.
Why do some models have only a few tests?
The totals grow as more people test and share. While there is little data, a total can rest on only a few tests, so read it with the counts beside it.
Find the cheaper model before your next invoice
- 10k free credits
- No card
- No API keys