Vision LLM
Benchmark

Extracting information from images.

VLMBench v1: an early public benchmark for VLM document reasoning.

5 models
550 questions
82,500 graded cells
v1 · 2026
VLMBench per-model results: a question with all 30 runs wrong and the source document below
Explore the full results →

What we measure

Each model

  • same image
  • same prompt
  • same question
×30

Analyze

  • Track consistency and accuracy
  • Observe the answer / solution space
  • Map error types

Results

Acceptable answers (exact + materially correct) out of 16,500 graded cells per model.

Recommend a model

Error Types

Errors were classified, here is the breakdown:

Consistency and accuracy

Suggest