Benchmarks
Independent, reproducible measurements of the knobs you can actually set on an OpenRouter request: models, providers, search engines, and tool budgets. Every score links to the configuration, costs, and telemetry behind it.
18 benchmarks2,283,353 task evaluationslast run Nov 14, 2026
Routers
Compare how model routers balance quality, cost and speed on each benchmark.
Media
Generated images, clips and speech, graded against the request and priced per output.
Image
Image prompts built to fail: depth, direction, counting, and text in the frame.
66 models
Value$0.007Recraft: Recraft V4.1 Flash
Speed2sRecraft: Recraft V4.1 FlashVideo
Six seconds of video, held to one duration and resolution across every model.
28 models
Value$0.050TBC: mischief 1Speed24sTBC: mischief 1Speech
One sentence, every voice: did the words come back, and was direction performed?
37 models
Value$0.000Mistral: Voxtral Mini TTS
Speed1sElevenLabs: Eleven Flash v2.5Memes
Edit a meme as an image or animate it as a clip, graded on whether the brief landed.
75 models
Value$0.009Black Forest Labs: FLUX 3 Image KleinSpeed7sBlack Forest Labs: FLUX.2 Klein 4B
Artifact generation
Text models take on unconventional tasks like creating drawings and playable games.
Sketch
Text models create drawings from prompts. The results are graded as images.
248 models
Value$0.000034Mistral: Mistral NemoSpeed2sMeta: Llama 3.2 3B Instruct
Games
Text models create playable games from a single brief in one attempt.
58 models
Value$0.000Crouton Alpha
Speed15sGoogle: Gemini 3.1 Flash Lite
Reasoning
GPQA Diamond
Graduate-level science questions that resist retrieval and reward careful reasoning.
150 modelslast run Nov 14, 2026
Quality96.5%Gemini 4 ArgonValue$0.021GPT-6.1 Sol
Speed20sClaude Sonnet 5.5
VGI-Bench
Multiple-choice questions about long videos: what was shown, said, or never happened.
67 modelslast run Nov 13, 2026
Quality66.2%Seed 1.6Value$0.039Seed 1.6Speed64sSeed 1.6
Search
BrowseComp
Hard-to-locate facts on the live web, scored on persistent multi-step research.
4 modelslast run Aug 18, 2026
Quality89.0%PerplexityClaude Opus 5 · highValue$0.99PerplexityClaude Opus 5 · highSpeed1.9mPerplexityClaude Opus 5 · highDeepSearchQA
Questions whose answers are lists, scored for exhaustive retrieval with no padding.
4 modelslast run Aug 18, 2026
Quality77.0%ParallelClaude Opus 5 · highValue$0.10PerplexityGPT-5.6 Luna · xhighSpeed1.6mPerplexityGPT-5.6 Luna · xhighHLE
Humanity's Last Exam as a search benchmark: expert questions answered with live search.
2 modelslast run Aug 17, 2026
Quality77.4%PerplexityClaude Opus 5 · highValue$0.16PerplexityClaude Opus 5 · highSpeed48sPerplexityClaude Opus 5 · highWideSearch
Fill an entire table; answer-item accuracy scores partial matches.
4 modelslast run Aug 18, 2026
Quality84.0%PerplexityGPT-5.6 Sol · highValue$0.063PerplexityGPT-5.6 Luna · xhighSpeed1.9mPerplexityGPT-5.6 Sol · high
Agents & tools
For usage-based views of the same models, see the model rankings and the full model list.