Write your prompts
The questions and tasks you actually use AI for.
Add each prompt exactly as you'd send it, up to 200 in a benchmark. Paste them in one at a time, import a batch as JSONL, JSON or CSV, or start from a template.
Build your own AI benchmarks. Add prompts, run them across the models you care about, rate the responses, and publish a shareable leaderboard.
Free trial included No credit card needed Your prompts stay private
Save a list of prompts you actually care about, pick a baseline of models you trust, and add new candidate models to compare against them. Each model gets up to ten attempts per prompt, you rate every response with stars, and Raq averages the scores into a public leaderboard table at your own URL. No need to trust someone else's benchmark. Build the one that matches your work.
Works in Claude, ChatGPT Work, Grok Bot and any AI app with connectors. Connect yours
Compare these AI models using our customer questions.
I’ll run the same questions through AI Benchmarker and put the responses together for your team to rate.
The questions and tasks you actually use AI for.
Add each prompt exactly as you'd send it, up to 200 in a benchmark. Paste them in one at a time, import a batch as JSONL, JSON or CSV, or start from a template.
Your own stars, an expected answer or an AI judge.
Rate the answers yourself after the run, give the answer you expect and check for it, or write the criteria and let a judge model score each answer. You can change any score the judge gives.
Up to 25 models in one run.
Mix companies, sizes and reasoning levels, and keep the models you trust as a baseline that runs every time. Each prompt can be repeated up to ten times to see how steady the answers are.
Score, cost and speed for every model.
See which model scored best, which was cheapest and which was quickest, then open any answer to read it. Publish the table as a leaderboard or share the run with a link.
Every run shows the AI usage it will take before any model is called, and a cap stops it if the cost runs past that.
Run a benchmark every day, week or month, or let it start on its own with just the new models when one that suits it appears.
Put a run on a public page with your name on it, or share one run with a link, a password and an end date.
Import a Promptfoo config as a benchmark, export one back, or start runs from your own scripts with an API key.
AI Benchmarker is included in every Raq.com plan. From $39 per user a month, with 75+ modules. AI work uses your plan's usage allowance.
Free trial included No credit card needed Your prompts stay private
We use optional cookies to understand how people find this site and to measure our ads.