ParseBench-2: Severity-aware document parsing benchmark

According to LlamaIndex Blog:

Background and Improvements

Traditional document parsing benchmarks often rely on brittle metrics that overpenalize harmless formatting changes while underpenalizing critical data errors, such as swapping digits in a number. ParseBench-2 addresses this by introducing severity-tuned scoring based on 330 expert-defined error types across six dimensions: text, formatting, tables, charts, layout, and forms. The benchmark corpus has been expanded from 2.1k to 4.3k real-world enterprise documents, including 1,025 charts and 500 form pages from 30 countries.

Performance Insights and OCRia-1.5

In the evaluation of 41 default configurations, LlamaParse Agentic Plus led overall with a score of 89.8, outperforming direct vision models and competing specialized parsers. Charts emerged as the widest capability gap among systems, with scores ranging from 1.2 to 92.6. Additionally, LlamaParse introduced OCRia-1.5, a new family of document parsing models powering its cost-effective and agentic parsing modes, aiming to deliver higher accuracy at lower processing costs.