HumanEval.org

Models

Everything we list: arena contestants with their latest official HumanEval ratings, plus models tracked only through external benchmarks — flagged “not yet evaluated” until they collect blind human votes.

ModelTypeProviderHumanEval ratingsLicenseIn / Out $/MtokContextReleased
CastleFlow v1.0not yet evaluatedText to speechAsync AI—proprietary— / ———
Claude Fable 5not yet evaluatedLLMAnthropic—proprietary$10.00 / $50.00—2026-06-09
Claude Fable 5.1LLMAnthropicunratedproprietary$10.00 / $50.001,000,0002026-09-01
Claude Opus 4.7not yet evaluatedLLMAnthropic—proprietary$5.00 / $25.00—2026-04-16
Claude Opus 4.8not yet evaluatedLLMAnthropic—proprietary$5.00 / $25.00—2026-05-28
Claude Opus 5LLMAnthropicunratedproprietary$5.00 / $25.001,000,0002026-07-24
Claude Sonnet 5LLMAnthropicunratedproprietary$2.00 / $10.001,000,0002026-06-30
DeepSeek V4 Pro (0813)openLLMDeepSeekunratedMIT$0.66 / $1.981,048,5762026-08-13
Dreamina Seedance 2.0 (720p)not yet evaluatedVideo generationByteDance—proprietary— / ———
Gemini 3.1 Flash Image (Nano Banana 2)not yet evaluatedImage generationGoogle—proprietary— / ———
Gemini 3.1 Pro Previewnot yet evaluatedLLMGoogle—proprietary$1.00 / $6.00—2026-02-19
Gemini 3.8 FlashLLMGoogleunratedproprietary$0.75 / $3.751,048,5762026-09-02
Gemini Omni Flashnot yet evaluatedVideo generationGoogle—proprietary— / ———
GLM-5.3not yet evaluatedLLMZ.ai—MIT (per LMArena listing; Epoch AI records API access only — open-weights status unconfirmed)$1.40 / $4.40—2026-08-14
GPT-5.4not yet evaluatedLLMOpenAI—proprietary$2.50 / $15.00—2026-03-05
GPT-5.6 LunaLLMOpenAIunratedproprietary$0.20 / $1.201,050,0002026-07-09
GPT-5.6 Solnot yet evaluatedLLMOpenAI—proprietary$4.00 / $20.00—2026-07-09
GPT-5.6 TerraLLMOpenAIunratedproprietary$2.00 / $12.001,050,0002026-07-09
GPT-6 AstraLLMOpenAIunratedproprietary$10.00 / $50.001,050,0002026-09-03
GPT Image 2not yet evaluatedImage generationOpenAI—proprietary— / ———
Grok 4.6LLMSpaceXAIunratedproprietary$2.00 / $6.00500,0002026-08-12
HappyHorse 1.0not yet evaluatedVideo generationAlibaba-ATH—proprietary— / ———
Hume Octavenot yet evaluatedText to speechHume AI—proprietary— / ———
Inworld TTSnot yet evaluatedText to speechInworld—proprietary— / ———
Inworld TTS MAXnot yet evaluatedText to speechInworld—proprietary— / ———
Kimi K3openLLMMoonshot AIunratedKimi K3 License (custom, open weights)$3.00 / $15.001,048,5762026-07-16
Llama 4 MaverickopenLLMMetaunratedLlama 4 Community License$0.20 / $0.701,048,5762025-04-05
MAI-Image-2.6not yet evaluatedImage generationMicrosoft AI—proprietary— / ———
MiniMax H3opennot yet evaluatedVideo generationMiniMax—MiniMax H3 Community License Agreement (per LMArena listing)— / ———
Muse Imagenot yet evaluatedImage generationMeta—proprietary— / ———
Papla P1not yet evaluatedText to speechPapla—proprietary— / ———
Qwen3.8 Max (0902)LLMAlibaba Qwenunratedproprietary$2.00 / $6.001,000,0002026-09-02
Reve 2.1not yet evaluatedImage generationReve—proprietary— / ———
Sora 2 Pronot yet evaluatedVideo generationOpenAI—proprietary— / ———

* provisional rating