Artifact generation benchmarks

Compare artifact quality across text models

Penguin with an umbrella

Why we chose this prompt: A penguin holding a red umbrella in the rain, as a 32×32 grid. Penguins are symmetric, so one wrong pixel shows, and umbrellas are curves, which a grid does not have.

Prompt

Pixel art of a penguin holding a red umbrella in the rain, drawn on a 32x32 grid with flat colors and no anti-aliasing

Sort by

Sort by

Author

Toughest critic: GPT-5.6 Sol−2.5 vs median
Biggest fan: Grok 4.6+0.6 vs median

GPT-5.4 Pro
$5.42733.8s
GPT-6 Astra Pro
$4.07799.1s
GPT-5.6 Sol Pro
$3.191054.1s
GPT-5.6 Terra Pro
$2.011078.0s
Claude Opus 4.6
$1.55853.3s
GPT-6 Astra
$0.91947.3s
GPT-5.6 Sol
$0.88456.3s
Claude Opus 5
$0.83378.6s
GPT-5.5
$0.79296.6s
Grok 4.20 Multi-Agent
$0.78130.5s
GPT-5.2
$0.72997.5s
GPT-5.4
$0.69546.6s
GPT-5.6 Terra
$0.68664.0s
GPT-5.2-Codex
$0.54619.4s
Kimi K3.1 Pro
$0.37472.8s
GPT-6 Sol
$0.36461.9s
GPT-5.1-Codex
$0.35522.3s
Gemini 3.1 Pro Preview
$0.29158.7s
Sonar Deep Research
$0.28227.3s
GLM 5.3
$0.28853.2s
GLM 5.1
$0.27699.4s
Ember-1
$0.24243.0s
GPT-6.1 Sol
$0.241050.2s
Mistral Medium 3.5
$0.24170.8s
Gemini 3.5 Flash
$0.22102.5s
Qwen3.8 2.4T A95B
$0.22193.0s
GPT-5.1-Codex-Max
$0.21117.5s
GPT-5.6 Luna Pro
$0.201078.1s
GPT-5.3-Codex
$0.20134.6s
Kimi K3
$0.19180.9s
Gemini 3 Flash Preview
$0.19322.9s
Grok 4.7
$0.18333.3s
GPT-5.1
$0.18244.7s
DeepSeek V4 Pro 0813
$0.18461.8s
Gemini 2.5 Pro Preview 05-06
$0.15325.1s
Gemini 3.8 Flash
$0.15196.2s
Gemini 4 Argon
$0.14170.2s
Grok 4.6
$0.14423.7s
Gemini 2.5 Pro
$0.14110.6s
Gemini 2.5 Pro Preview 06-05
$0.14243.6s
Gemini 3.7 Flash
$0.13207.5s
GPT-5
$0.12152.1s
Claude Opus 4
$0.09920.8s
Gemini 3.1 Flash Lite
$0.096143.5s
Gemini 3.6 Flash
$0.093105.5s
Inkling
$0.090172.0s
GLM 5
$0.089302.6s
Claude Opus 4.1
$0.08958.8s
DeepSeek V4 Pro 0423
$0.085442.6s
Seed 2.1 Turbo
$0.085513.8s
Hy4 preview
$0.073414.8s
Muse Spark Open
$0.071108.3s
Sonar Pro
$0.07072.7s
Kimi K2.7 Code
$0.06992.8s
GLM 5.2
$0.061319.3s
GPT-5 Mini
$0.061341.9s
GLM 5V Turbo
$0.059983.4s
Qwen3 VL 235B A22B Thinking
$0.057243.7s
GLM 5.3 FlashX Legal
$0.049280.5s
MiniMax M3
$0.049160.5s
Qwen3.6 27B
$0.049703.1s
Claude Opus 4.5
$0.04717.3s
Qwen3 235B A22B Thinking 2507
$0.044420.6s
Palmyra X5
$0.04388.2s
Grok 4.5
$0.04280.4s
GPT-4o
$0.04214.9s
GPT-4
$0.0415.9s
Gemini 3.5 Flash Lite
$0.04141.8s
Glimpse
$0.03752.4s
Seed-2.0-Code
$0.036161.6s
GPT-5.6 Luna
$0.035327.6s
MiniMax M2.7
$0.035612.1s
MiMo-V2.6-Pro
$0.033665.0s
MiniMax M2.1
$0.033280.7s
Kimi K2.5
$0.032279.3s
Sonar Pro Search
$0.03216.8s
Qwen3.8 27B
$0.03282.3s
Claude Sonnet 4.5
$0.03117.9s
GPT-4o
$0.0317.9s
MiniMax M2
$0.029247.2s
Inkling Small
$0.029265.9s
Claude Sonnet 4
$0.02814.1s
GLM 4.7
$0.028231.9s
MiniMax M2.5
$0.027261.3s
Qwen3.6 35B A3B
$0.02792.4s
DeepSeek V4.1 Flash
$0.02665.2s
Muse Glimmer 30B
$0.025290.5s
GLM 5.3 FlashX
$0.025132.9s
Grok Build 0.1
$0.02399.2s
DeepSeek V4 Flash Vision Exp
$0.02348.9s
Grok 4.3
$0.02366.1s
Kimi K2.6
$0.02357.1s
GLM 4.5V
$0.021603.3s
R1 0528
$0.021449.2s
Qwen3.5-122B-A10B
$0.021117.0s
MiniMax M3.1 Flash
$0.02176.2s
Mercury 2
$0.02126.7s
Step 3.7 Flash
$0.020113.3s
Inkling 2 Small
$0.02022.3s
Mercury 3
$0.019139.7s
GPT-6 Luna
$0.019482.6s
Seed-2.0-Lite
$0.01996.8s
Qwen3 Next 80B A3B Thinking
$0.018152.4s
Hy3
$0.017354.3s
Beam 1.0 501B A23B
$0.01727.6s
Nemotron 3 Ultra
$0.01744.2s
Qwen3.5-27B
$0.01785.0s
Seed-2.0-Mini
$0.016301.1s
Mistral Large 4
$0.01678.0s
Sonar Reasoning Pro
$0.01677.5s
DeepSeek V4.1 Flash 1017
$0.015120.5s
GLM 4.5
$0.014162.2s
Qwen3.5-35B-A3B
$0.013101.2s
Step 3.5 Flash
$0.012589.1s
GLM 4.6V
$0.012369.9s
Qwen3.5 397B A17B
$0.012121.5s
Qwen3 VL 30B A3B Thinking
$0.012256.5s
Particle-1
$0.01120.6s
Seed 1.6
$0.011124.7s
Mistral Large 2407
$0.01016.5s
Nemotron 3.5 Super VL
$0.00926.7s
Ling 3.1 Flash
$0.009335.4s
DeepSeek V4 Flash 0731
$0.009977.0s
GLM 5.3 Flash
$0.009247.0s
GPT-5 Nano
$0.008274.4s
Mixtral 8x22B Instruct
$0.0086.5s
GPT-4.1
$0.00822.8s
Nova Premier 1.0
$0.00713.7s
Sonar
$0.00710.9s
Pareto 26.10
$0.00657.6s
Nova Pro 1.0
$0.0067.3s
LFM2.5-24B-A2B
$0.00639.3s
Mistral Small 4
$0.00550.3s
Gemini 2.5 Flash
$0.00511.7s
GPT-4o
$0.0053.2s
Claude 3 Haiku
$0.00522.6s
GLM 4.6
$0.00525.1s
Qwen3.8 Flash
$0.00564.7s
Claude Haiku 4.5
$0.0055.1s
Grok 4.20
$0.00415.6s
gpt-oss-20b
$0.004236.1s
Ling 3.0 Flash Sante
$0.004112.8s
Perceptron Mk1.5
$0.0047.5s
Nemotron 3.5 Lightning
$0.00440.6s
Magnum v4 72B
$0.00436.8s
Granite 4.2 8B
$0.003130.1s
Nova 2 Lite
$0.0036.6s
Mistral Medium 3.1
$0.0038.7s
GLM 4.7 Flash
$0.003451.0s
Devstral 2 2512
$0.0039.0s
Qwen3 30B A3B
$0.00238.4s
Mistral Large 3 2512
$0.00216.6s
GPT-4.1 Mini
$0.00220.4s
Solar Mini 4
$0.002131.2s
Qwen3 Coder Next
$0.0028.7s
GPT-3.5 Turbo
$0.00212.4s
Seed 1.6 Flash
$0.00235.9s
Qwen3 VL 235B A22B Instruct
$0.00231.3s
Qwen3 Coder 480B A35B
$0.0023.9s
Mistral Medium 3
$0.0024.5s
Perceptron Mk1
$0.00120.7s
LFM3-1.7B
$0.001535.7s
DeepSeek V3.1
$0.00152.8s
Qwen3 VL 8B Instruct
$0.00147.5s
Qwen3 32B
$0.001189.8s
Ling 3.0 Flash
$0.00143.4s
gpt-oss-120b
$0.00192.7s
Qwen3 Next 80B A3B Instruct
$0.0016.3s
GPT-4o-mini
$0.00130.7s
DeepSeek V3.1 Terminus
$0.00157.3s
Uncensored
$0.0019.9s
Qwen3.5-9B
$0.001437.8s
Nemotron 3 Nano 30B A3B
$0.00142.0s
Qwen3 14B
$0.00176.8s
ERNIE 4.5 VL 424B A47B
$0.00117.6s
gpt-oss-safeguard-20b
$0.0013.9s
Qwen3 VL 30B A3B Instruct
$0.0009312.5s
Solar Pro 4
$0.0009241.3s
DeepSeek V3 0324
$0.0008913.4s
Gemini 2.5 Flash Lite
$0.000874.7s
MiMo-V2.6-Flash
$0.0008120.8s
DeepSeek V3.2
$0.0007910.2s
Cydonia 24B V4.1
$0.0007317.8s
Gemma 3 12B
$0.0007114.2s
Hermes 3 405B Instruct
$0.0006921.3s
DeepSeek V3
$0.0006926.9s
Mistral Small 3.1 24B
$0.0006829.4s
Qwen3 235B A22B Instruct 2507
$0.000675.5s
Mercury 2.5
$0.000659.7s
WizardLM-2 8x22B
$0.0005824.9s
Qwen3 30B A3B Instruct 2507
$0.0005484.6s
Saba
$0.000514.6s
Llama 4 Maverick
$0.000519.0s
Llama 3.3 70B Instruct
$0.0005124.2s
Llama 3.1 70B Instruct
$0.0004930.0s
DeepSeek V3.2 Exp
$0.0004697.1s
Ripple 65B Preview
$0.0004414.7s
Gemma 4 31B
$0.0003941.3s
Gemma 4 26B A4B
$0.0003867.9s
Qwen2.5 VL 72B Instruct
$0.000355.4s
GPT-4.1 Nano
$0.000315.4s
Nova Lite 1.0
$0.000275.9s
Ministral 3 8B 2512
$0.0002315.4s
Ministral 3 14B 2512
$0.000239.7s
Gemma 3 27B
$0.0001548.4s
Ministral 3 3B 2512
$0.000155.6s
Hy-MT2-7B
$0.0000922.8s
Phi 4
$0.0000767.7s
Mistral Small 3
$0.0000611.5s
Llama 3.2 3B Instruct
$0.0000552.0s
Granite 4.0 Micro
$0.00004510.6s
Mistral Nemo
$0.0000432.8s
Hy-MT2-1.8B
$0.0000271.2s