Mistral AI in Converting a complex PDF into clean Markdown with a hosted API
Scenario-level performance from current published Results.
10 of 12 scenarios published
Where Mistral AI failed
2 of 9 graded scenarios. Open a Result to see the conversation and the evidence.
A document containing a code block
Mistral AI returned code without fences and flattened its nested indentation when converting the PDF to Markdown. The Python code was no longer runnable, and the shell command lost its continuation indentation.
View Result →An ordinary digital text document
Mistral AI did not keep ordinary digital text complete and unchanged. The returned Markdown preserves the body prose, but page furniture lands inside the text at four page breaks, and the raw page markdown shows the same split.
View Result →How Mistral AI performed
Open a capability to explore its scenarios. Each row reports the test set in its published Result.
Code Extraction1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A document containing a code block | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedMistral AI returned code without fences and flattened its nested indentation when converting the PDF to Markdown. The Python code was no longer runnable, and the shell command lost its continuation indentation. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Equations & Mathematical Notation1 pass
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A document containing mathematical equations | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedMistral AI kept the mathematical equations as LaTeX math in the Markdown for this PDF. The returned Markdown preserved all six display equations as math blocks with tags, kept inline symbols inside math delimiters, and carried through the page-break text and running heads. The page-by-page comparison sheets matched the source pages shown there. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Figures & Charts1 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A document with figures and captions | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedMistral AI preserved the figure at its source-page position and kept the caption directly below it. The Markdown places `` between the two surrounding paragraphs, with the Figure 1 caption immediately beneath it. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A document with a data chart | No published result | ||
Heading & Section Structure1 pass1 no Result yet
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| Footnotes at the bottom of the page | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedMistral AI keeps footnotes out of the body flow and links each marker to its note. The Markdown output places 13 footnotes after the page bodies in source order, and the 26 marker occurrences each appear once inline and once at their notes, so the association is preserved. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A document with styled headings and subheadings | No published result | ||
Reading Order & Layout1 pass
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A page laid out in multiple columns | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedMistral AI read the two-column page down each column in turn rather than across the page. On the three pages, the left and right columns stayed in order without interleaving. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Scanned Document OCR2 pass
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A cleanly scanned document | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedMistral AI accurately recovered text from a clean scanned document. It placed the results table’s first data row, the sampling point, in the Markdown header without losing text. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A document mixing digital and scanned pages | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedMistral AI processed the scanned page in a mixed digital and scanned document rather than skipping it. The page appeared in document order, with tables in the same style as the digital pages. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Table Extraction1 pass1 different test set
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| A simple, clearly formatted table | 1 Pass0 Fail0 Not gradable 1 of 1 test case passedWhat happenedMistral AI preserved a simple, clearly formatted table: the returned Markdown kept the four headers in order, retained eight data rows, and matched all 32 data cells against the source. The table stayed a single pipe table, and the comma inside "Metals suite, eleven elements" did not split the cell. | 1/1 assessed1/1 gradablePublished test set | View Result → |
| A table that continues across a page break | Different test setPublished Result available, but not included in this comparison. | View Result → | |
Text Fidelity1 fail
Capability in this benchmark →
| Scenario | Published outcomes | Test coverage | Result |
|---|---|---|---|
| An ordinary digital text document | 0 Pass1 Fail0 Not gradable 1 of 1 test case failedWhat happenedMistral AI did not keep ordinary digital text complete and unchanged. The returned Markdown preserves the body prose, but page furniture lands inside the text at four page breaks, and the raw page markdown shows the same split. | 1/1 assessed1/1 gradablePublished test set | View Result → |
Reading these Results
Published evidence and test coverage answer different questions.
Which scenarios have a Result?
A published Result is public evidence for this tool on one scenario. “No published result” does not say whether testing has taken place.
What does each Result cover?
Assessed includes Pass, Fail and Not gradable. Gradable includes Pass and Fail. Both use the pinned test count in that published Result.
Out of 12 scenarios
The summary counts each of the 12 scenarios once. A scenario fails when any of its published test cases failed. “No published Result yet” means nothing is public for that scenario; it does not say whether Mistral AI was tested on it.