Artifact generation benchmarks

Compare artifact quality across text models

Lighthouse at night

Why we chose this prompt: A lighthouse on a rock at night, as a 32×32 grid. The sky, sea, rock, and tower have to stack in the right order, and the model can optionally include a beam of light.

Prompt

Pixel art of a lighthouse on a rock at night, drawn on a 32x32 grid with flat colors and no anti-aliasing

Sort by

Sort by

Author

Toughest critic: GPT-5.6 Sol−1.2 vs median
Biggest fan: Gemini 3.1 Pro Preview+0.7 vs median

GPT-5.6 Sol Pro
$2.62526.6s
GPT-5.6 Terra Pro
$1.64678.2s
GPT-5.6 Sol
$0.75349.5s
Claude Opus 4.8
$0.75389.3s
GPT-6 Astra
$0.74971.3s
Grok 4.20 Multi-Agent
$0.71152.9s
Sonar Deep Research
$0.71436.3s
GPT-6 Sol Pro
$0.67620.7s
GPT-5.4
$0.58515.3s
Claude Opus 5
$0.57277.2s
GPT-5.5
$0.55369.8s
GPT-5.6 Terra
$0.49445.0s
GPT-5.2
$0.46661.4s
GPT-5.2-Codex
$0.35360.2s
Gemini 3.1 Pro Preview
$0.32168.9s
Kimi K3
$0.32377.4s
GPT-5.1-Codex
$0.32260.5s
GPT-6 Sol
$0.29411.2s
GPT-5.3-Codex
$0.27160.0s
Gemini 3.5 Flash
$0.26144.1s
Ember-1
$0.24245.3s
Kimi K3.1 Pro
$0.22291.2s
Gemini 3 Flash Preview
$0.19337.1s
Gemini 4 Argon
$0.19227.1s
Mistral Medium 3.5
$0.18125.7s
Muse Spark Open
$0.1781.6s
Qwen3.8 2.4T A95B
$0.17284.3s
GPT-6.1 Sol
$0.16496.6s
GPT-5.1-Codex-Max
$0.15246.6s
GLM 5.2
$0.14218.1s
Grok 4.7
$0.14241.9s
Gemini 2.5 Pro Preview 06-05
$0.12102.8s
DeepSeek V4 Pro 0813
$0.12357.9s
Gemini 3.7 Flash
$0.12195.4s
Gemini 2.5 Pro
$0.10191.8s
Gemini 3.6 Flash
$0.10147.3s
GPT-5.1
$0.097153.1s
GPT-5
$0.096132.6s
Gemini 3.1 Flash Lite
$0.095158.7s
Grok 4.6
$0.091268.4s
Seed 2.1 Turbo
$0.091497.6s
GLM 5 Turbo
$0.090541.0s
Claude Opus 4
$0.08719.8s
GPT-5.1-Codex-Mini
$0.086274.8s
MiniMax M3
$0.080493.4s
Inkling
$0.07964.8s
Hy4 preview
$0.077411.0s
Claude Opus 4.1
$0.07445.0s
GLM 5V Turbo
$0.061381.3s
Qwen3.6 27B
$0.061403.4s
Glimpse
$0.05889.5s
Step 5 Preview
$0.053136.0s
GPT-5 Mini
$0.053727.7s
GPT-5.6 Luna
$0.052417.7s
Seed-2.0-Code
$0.050241.5s
MiniMax M2.1
$0.049348.6s
GLM 5
$0.049244.9s
Qwen3.8 27B
$0.049246.3s
Claude Opus 4.5
$0.04318.6s
Sonar Pro Search
$0.04233.4s
Kimi K2.7 Code
$0.041213.1s
GLM 5.3 FlashX Legal
$0.040212.5s
Grok 4.5
$0.03780.8s
Gemini 3.5 Flash Lite
$0.03645.3s
GPT-6 Luna Pro
$0.035675.3s
Mistral Large 4
$0.034240.2s
MiniMax M2.5
$0.030201.3s
Qwen3 235B A22B Thinking 2507
$0.02942.9s
Sonar Pro
$0.02620.5s
Claude Sonnet 4.5
$0.02515.8s
GLM 4.5V
$0.024245.6s
Qwen3 VL 235B A22B Thinking
$0.022102.8s
Grok Build 0.1
$0.021102.5s
GLM 5.3 FlashX
$0.020107.3s
Qwen3.5 397B A17B
$0.02061.2s
R1 0528
$0.020391.5s
Mercury 2
$0.02052.4s
MiniMax M2
$0.019174.8s
R1
$0.018268.6s
DeepSeek V4.1 Flash
$0.018877.4s
DeepSeek V4 Flash Vision Exp
$0.018126.2s
Inkling 2 Small
$0.01824.0s
Claude Sonnet 4
$0.0178.3s
GPT-4
$0.0162.9s
DeepSeek V4.1 Flash 1017
$0.016597.8s
Seed-2.0-Mini
$0.016308.3s
Grok 4.3
$0.01569.9s
GPT-6 Luna
$0.015389.3s
Seed-2.0-Lite
$0.01477.5s
Beam 1.0 501B A23B
$0.01444.9s
Sonar Reasoning Pro
$0.014136.8s
MiMo-V2.6-Pro
$0.013240.7s
Nemotron 3 Ultra
$0.01343.4s
Muse Glimmer 30B
$0.01393.1s
GLM 4.5
$0.013115.8s
Qwen3.5-122B-A10B
$0.01334.2s
Particle-1
$0.01218.5s
GLM 4.6V
$0.012435.4s
Inkling Small
$0.01232.3s
Qwen3 Next 80B A3B Thinking
$0.01254.7s
MiMo-V2.6-Flash
$0.011643.0s
Kimi K2.6
$0.01136.2s
Qwen3.5-35B-A3B
$0.01043.3s
Step 3.7 Flash
$0.01061.8s
GLM 4.7
$0.01027.5s
MiniMax M3.1 Flash
$0.010129.4s
Nemotron 3.5 Lightning
$0.009114.8s
GLM 4.5 Air
$0.009123.3s
Mistral Large 2407
$0.00914.0s
GPT-5 Nano
$0.009377.5s
Seed 1.6
$0.00985.4s
Qwen3.5-27B
$0.00836.6s
Qwen3 VL 30B A3B Thinking
$0.007115.7s
Mercury 3
$0.00752.5s
Perceptron Mk1.5
$0.00715.0s
Claude Haiku 4.5
$0.0077.6s
gpt-oss-120b
$0.00720.9s
Kimi K2.5
$0.007104.8s
GPT-4.1
$0.0075.9s
Sonar
$0.00713.4s
Pareto 26.10
$0.00667.2s
Nova Premier 1.0
$0.00612.7s
Qwen3 Coder Next
$0.00617.8s
LFM2.5-24B-A2B
$0.00530.2s
Mistral Small 4
$0.00546.1s
GPT-4o
$0.0052.9s
MiniMax M2.7
$0.00590.8s
Palmyra X5
$0.0059.4s
DeepSeek V4 Flash 0731
$0.0041131.2s
Hy3
$0.004297.1s
Mixtral 8x22B Instruct
$0.0044.2s
Ling 3.1 Flash
$0.004156.0s
GPT-4o
$0.0043.2s
GPT-3.5 Turbo 16k
$0.0049.8s
Step 3.5 Flash
$0.004152.7s
Nemotron 3.5 Super VL
$0.00425.5s
Qwen3.8 Flash
$0.00446.2s
Grok 4.20
$0.00312.7s
Magnum v4 72B
$0.00331.7s
Nova 2 Lite
$0.0035.2s
Qwen3 32B
$0.003540.0s
Qwen3.6 35B A3B
$0.00327.1s
Mistral Large 3 2512
$0.00318.4s
Seed 1.6 Flash
$0.00351.6s
Relace Search
$0.0033.1s
Solar Mini 4
$0.00297.8s
GLM 4.6
$0.00251.4s
MiMo-V2.5
$0.002113.3s
Gemini 2.5 Flash
$0.0023.5s
GPT-3.5 Turbo
$0.00211.0s
Qwen3 30B A3B
$0.00229.7s
GPT-4.1 Mini
$0.00212.8s
Nova Pro 1.0
$0.0023.1s
DeepSeek V3.1 Terminus
$0.00256.7s
Nemotron 3 Super
$0.00269.1s
Granite 4.2 8B
$0.00248.2s
Mistral Medium 3
$0.0014.3s
Qwen3 Coder 480B A35B
$0.0018.2s
DeepSeek V3.1
$0.00155.8s
Devstral 2 2512
$0.0014.5s
Perceptron Mk1
$0.00119.3s
Qwen3 Next 80B A3B Instruct
$0.00115.3s
Mistral Medium 3.1
$0.0013.8s
Qwen3 235B A22B Instruct 2507
$0.00177.8s
ERNIE 4.5 VL 424B A47B
$0.0009318.0s
DeepSeek V3
$0.0009124.4s
gpt-oss-safeguard-20b
$0.000883.4s
Qwen3 14B
$0.0008451.4s
Ling 3.0 Flash VL
$0.000886.9s
Claude 3 Haiku
$0.000733.7s
Hermes 3 70B Instruct
$0.000727.8s
Ripple 65B Preview
$0.0006923.6s
DeepSeek V3 0324
$0.0006815.1s
Saba
$0.000655.5s
DeepSeek V3.2 Exp
$0.0006160.3s
Mercury 2.5
$0.000585.1s
DeepSeek V3.2
$0.0005553.5s
Qwen3 VL 235B A22B Instruct
$0.0005340.7s
Qwen3 VL 30B A3B Instruct
$0.0005230.8s
Gemini 2.5 Flash Lite
$0.000523.6s
WizardLM-2 8x22B
$0.0005116.8s
Cydonia 24B V4.1
$0.0004515.1s
Llama 4 Maverick
$0.000427.0s
Hermes 3 405B Instruct
$0.000416.1s
Gemma 4 26B A4B
$0.000378.6s
Mistral Small 3.1 24B
$0.0003512.1s
Qwen3 30B A3B Instruct 2507
$0.0003354.5s
Gemma 4 31B
$0.0003221.0s
Gemma 3 27B
$0.0002916.2s
GPT-4.1 Nano
$0.000286.4s
Gemma 2 27B
$0.000254.5s
Hy-MT2-30B-A3B
$0.0002310.8s
Ministral 3 14B 2512
$0.000211.3s
Mistral Small 3.2 24B
$0.000219.5s
GPT-4o-mini
$0.00026.1s
Ministral 3 8B 2512
$0.000168.7s
Qwen2.5 7B Instruct
$0.0001510.1s
Phi 4
$0.0001311.8s
Ministral 3 3B 2512
$0.000137.1s
Llama 4 Scout
$0.00009411.6s
Hy-MT2-7B
$0.0000933.6s
Mistral Small 3
$0.00009124.0s
Gemma 3 4B
$0.00005323.6s
Nova Micro 1.0
$0.0000511.5s
Mistral Nemo
$0.00003812.2s
Granite 4.0 Micro
$0.0000358.4s
Hy-MT2-1.8B
$0.0000221.1s