Converting a complex PDF into clean Markdown with a hosted API
This benchmark helps buyers choose a hosted API for PDF-to-Markdown conversion.
Benchmark overview
What is included and excluded
Evaluates whether a hosted API turns complex PDFs into faithful, usable Markdown without dropping, reordering, or inventing content.
A reader learns whether the API preserves the document's text, order, structure, tables, figures, scanned pages, equations, and code, and whether it handles failures honestly when the source cannot be represented cleanly.
In scope
- Conversion of complex, real-world PDFs into faithful Markdown.
- Preservation of text, reading order, headings, section structure, tables, figures, charts, scanned text, equations, and code.
- Honest fallback when a source element cannot be expressed cleanly in Markdown.
Out of scope
- Structured field extraction.
- Web-page conversion.
- Form filling.
- Handwriting.
- Document classification and routing.
- PDF creation or editing.
Participating tools
How each tool did on the 12 scenarios in this benchmark. Each cell is one scenario, in the same order for every tool and grouped by capability; Pass and Fail come from published Results.
| Tool | Scenario results · 12 scenarios in 8 capabilities | Passed | Failed | Explore |
|---|---|---|---|---|
| Mistral AI | 7of 9 graded | 2 | View tool → | |
| Adobe PDF Extract API | 6of 8 graded | 2 | View tool → | |
| Firecrawl | 3of 8 graded | 5 | View tool → | |
| Extend AI | 3of 6 graded | 3 | View tool → | |
| Nutrient.io | 1of 6 graded | 5 | View tool → | |
| Reducto | 3of 6 graded | 3 | View tool → | |
| Upstage AI | 3of 5 graded | 2 | View tool → | |
| Landing AI | Failed: A document with a data chart | 1of 2 graded | 1 | View tool → |
| Unstructured | 1of 2 graded | 1 | View tool → | |
| GPT-5.6 terra | 1of 1 graded | 0 | View tool → | |
| LlamaParse | 1of 1 graded | 0 | View tool → | |
| 2 more toolsin the roster | No published Results yet for PDF Vector · Tensorlake.Each tool gets its own row here once its first Result is published. | |||
Capabilities & scenarios
12 scenarios grouped by 8 capabilities. Open a group to explore its scenarios in this benchmark.
Text Fidelity1 scenario
Ordinary digital text survives completely and correctly in the Markdown output.
Capability in this benchmark → · Global definition →
- An ordinary digital text document6 tools with published Results
Reading Order & Layout1 scenario
The source reading sequence survives the spatial-to-linear conversion without the columns being interleaved or reordered.
Capability in this benchmark → · Global definition →
- A page laid out in multiple columns5 tools with published Results
Heading & Section Structure2 scenarios
Document structure survives as real Markdown headings and sections, with nesting and footnotes kept in the right relationship.
Capability in this benchmark → · Global definition →
- A document with styled headings and subheadings4 tools with published Results
- Footnotes at the bottom of the page4 tools with published Results
Table Extraction2 scenarios
Rows, columns, headers, and values survive as a table with the right relationships intact.
Capability in this benchmark → · Global definition →
- A simple, clearly formatted table6 tools with published Results
- A table that continues across a page break2 tools with published Results
Figures & Charts2 scenarios
Visual content survives with its position, caption or title, and a text-reachable trace for later use.
Capability in this benchmark → · Global definition →
- A document with figures and captions2 tools with published Results
- A document with a data chart4 tools with published Results
Scanned Document OCR2 scenarios
Text that exists only as pixels is recovered into usable output instead of being silently lost.
Capability in this benchmark → · Global definition →
- A cleanly scanned document8 tools with published Results
- A document mixing digital and scanned pages5 tools with published Results
Equations & Mathematical Notation1 scenario
Mathematical notation survives as math markup or an honest fallback, without being turned into misleading prose.
Capability in this benchmark → · Global definition →
- A document containing mathematical equations4 tools with published Results
Code Extraction1 scenario
Code survives as a preformatted fenced block with its whitespace and line structure intact.
Capability in this benchmark → · Global definition →
- A document containing a code block6 tools with published Results
Results overview
Current published evidence in this benchmark.
Publication availability is separate from test coverage and unpublished research progress.
How the benchmark works
A public summary of the evaluation method. The same defined scope and evidence standard apply to every tool assessed under this version.
One test case, one tool
One test case is scored for one tool at a time.
Pass, fail, or partial
Each run ends pass, fail, or partial. A missing capability scores 0 and stays in the arithmetic — a product must not rank higher by having less product. not_applicable, not_measured and blocked (with the reason, never a 0) are distinct states.
Versioned scenarios and resources
Use versioned scenarios and registered resource versions so runs stay comparable.
Recorded stimulus and complete output
Record the input stimulus, the complete output, and the relevant registered references.
Resources and fixtures
The registered material and systems that create a consistent test environment for this benchmark.
PDF to Markdown test documents
A shared pinned PDF corpus used as input for both PDF-to-Markdown benchmark versions.