AI Demos Research — the structured-intelligence platform. Every verdict on these pages opens to the execution behind it.
Graded 24 October 2026

Does PyMuPDF4LLM keep styled headings as real Markdown heading levels?

PyMuPDF4LLM kept the heading hierarchy in this PDF-to-Markdown run. It produced 15 #, 9 ## and 2 ### lines in source order; the three body-sized title lines were emitted as ####, but that false positive did not change the real heading structure.

1 of 1 test case passed

Every test case in this scenario has a result.

Pass rate100%1 of 1 gradable
Coverage1 of 1test case with a result
1PassDid everything it was expected to do.
0FailNo test case failed.
0Not gradableEvery result here could be graded.
0UntestedEvery test case in this scenario has a result.
The pass rate is a summary. The evidence is the test case below: what we sent, what we checked, what the tool returned, and the proof.

The test case

Each test case is judged on its own: Pass, Fail, Not gradable, or Untested. The scenario result above is based on this test case.

Styled headings — hierarchy preservedPassEvidence
What we sent
InputDocument (PDF)
Source PDF with three heading tiers over body text
Open the PDF ↗
What the tool returned
Markdownthe whole file · copied lines 1–63 marked
#### Kirkhollow Analytical Laboratories Ltd 

#### Sample Reception and Handling 

#### Procedure KAL-SP-06 · Revision 4 · Effective 1 March 2026 

# Scope 

This procedure covers the receipt, identification, storage and disposal of samples submitted to the laboratory for chemical analysis. It applies to everything received at the Braithe Road reception, including samples delivered by courier and samples brought in by clients. It does not cover samples taken by our own field staff, which are dealt with in KAL-SP-11. 

# Reception 

Samples are received at reception between 08:00 and 17:30 on weekdays and between 09:00 and 12:00 on Saturdays. Outside those hours clients who hold a key may use the refrigerated drop box at the rear entrance. 

## Chain of custody 

Every delivery arrives with a completed submission form. Reception staff check that the form is signed, that the number of containers matches the number listed on it, and that each container carries a legible client reference. The form is signed and timed on receipt and the top copy is returned to the person making the delivery. 

## Condition on arrival 

Reception staff record the condition of each container on the submission form before it leaves the reception bench. Anything recorded there that falls outside the limits below goes to the duty chemist before the sample is booked in. 

### Temperature 

Chilled samples. Where the submission form specifies chilled transport, the temperature of the transport box is measured and written on the form. A box above 8 degrees is accepted, marked, and referred to the duty chemist, who decides whether the analysis can still be reported. A box above 15 degrees is not accepted for microbiological work at all. 

### Container integrity 

Containers that are cracked, leaking, or filled above the shoulder are photographed on the reception bench and set aside, and the client is telephoned the same day. A leaking container is not opened at reception. 

# Storage 

## Refrigerated storage 

Samples for chemical analysis are held at 4 degrees in the reception cold room and are racked by the day of receipt, not by client. Racks are cleared every Friday afternoon. 

## Frozen storage 

Samples that require frozen storage are held at minus 20 degrees in the chest freezer in room 4. Nothing goes into the freezer without a completed freezer label, and nothing is stacked above the load line marked inside the cabinet. 

# Retention and disposal 

Analysed samples are kept for twenty-eight days after the report is issued and are then disposed of by the route set out in KAL-SP-19. Samples held longer at a client's request are labelled with the agreed retention date and reviewed monthly by the laboratory manager. 

# Records 

Submission forms, condition records and the reception log are kept for six years. The reception log is a bound book: entries are not erased, and a correction is made by a single line through the original, initialled and dated. 

# Responsibilities 

The laboratory manager owns this procedure. Reception staff carry it out; the duty chemist decides anything it refers to them; and the quality manager audits it twice a year against the reception log and the submission forms. 

## Reception staff 

Reception staff check every delivery against its form, record the condition of each container, book the submission in, and put the samples into the right storage within thirty minutes of accepting them. They do not decide whether a sample outside the limits above can be analysed, and they are not asked to. 

## The duty chemist 

The duty chemist is named on the board in reception for each working day. They decide whether a sample that arrived outside the limits can be reported and on what terms, record the decision on the submission form and initial it, and tell the client where the decision changes what the report will say. 

# Identification and booking in 

## The laboratory number 

Numbers run in one sequence for the whole laboratory and are never reused, including where a submission is rejected after being numbered. The number and the year are written on the container as well as on the form, because a container that has lost its label is a container nobody can report. 

## Splitting a submission 

Where determinations on one sample must be done in different sections, the sample is split at reception and each part carries the laboratory number with a suffix. The split is recorded on the form with the volume in each part. 

## Sub-samples for microbiology 

A sub-sample for microbiological work is taken first, before any other part is poured, and is taken in the reception cabinet rather than on the open bench. Where the volume received is too small to allow that, the microbiological determinations are refused rather than taken second. 

# Storage capacity 

The reception cold room holds forty racks and the freezer sixty labelled boxes. Where either is full, reception tells the duty chemist before accepting anything further into it. A sample put into the wrong storage is worse than a sample refused: the refusal is visible and the wrong storage is not. 

# Weekly clearance 

Racks are cleared every Friday afternoon against the booking-in record. A sample still in the cold room after its retention date is disposed of by the route in KAL-SP-19 and the disposal is logged. A sample in the cold room with no record at all is not disposed of: it goes to the duty chemist, because an unrecorded sample is a failure somewhere earlier in this procedure. 

# Transport boxes and returns 

Transport boxes belonging to the laboratory are emptied, wiped and returned on the next round to the client they came from. Boxes belonging to a client are emptied and returned unwashed unless the client asks otherwise. Ice packs are refrozen and reused; a leaking ice pack is thrown away rather than refrozen. 

# Out-of-hours deliveries 

The refrigerated drop box is checked at 08:00 every weekday and at 09:00 on Saturdays. Anything in it is booked in as arriving at the time of the check and not at the time it was left, and clients are told that is the practice when a key is issued. The box is not checked on Sundays or public holidays, and clients are asked not to use it on the day before one. 

# Deviations 

A deviation from this procedure is recorded on the submission form and in the reception log, with the reason and the name of whoever authorised it. Deviations are reviewed monthly by the laboratory manager and again at each audit. A deviation that has happened three times is a sign that the procedure is wrong rather than the staff, and is raised as such. 

# Complaints 

A client complaint about the condition or the handling of a sample goes to the laboratory manager on the day it is received and is acknowledged within one working day. The reception log, the submission form and the condition record are copied into the complaint file before anything else is done, because they are the only evidence of what arrived. 

# Training 

Nobody works reception unsupervised until they have been through this procedure with the laboratory manager, worked two full days alongside someone who has, and had their first ten submissions checked. Training is recorded on the individual's file, and the record says who signed each stage off. 

# Revision history 

Revision 4 adds the drop box and the fifteen-degree limit for microbiological work, and moves the retention period from twenty-one days to twenty-eight. Revision 3 followed the 2024 audit and rewrote the condition records. Revisions 1 and 2 are held in the controlled document file and are not in circulation. 

Markdown output with heading levels and title block

Copied from Output file (Markdown), lines 1–63 · an excerpt

Open the whole Markdown file ↗
Expected vs. Found
✓Expected: Headings become Markdown headings with their hierarchy preserved, not plain or bold text.Found: The output kept the three heading tiers in order as #, ##, and ### lines; no heading-sized source line stayed plain or bold.
Supporting proof
Proof 1Screenshot
Capture with console, Markdown excerpt, and page preview
Open original ↗
Why this result

The PDF has three heading tiers above body text, and the output reproduces them as 15 #, 9 ##, and 2 ### lines in the same order, with no heading-sized source line left as plain or bold text. The extra #### title lines are body-sized, so they do not break the required hierarchy.

Also observed in this test case

Body-sized title block. The three title lines are body-sized, yet the output renders them as #### before the first real heading.

Absolute outline start. A consumer reading levels absolutely would see the outline start at H4 rather than H1.

Tested by Vamshidhar Rao Thumula · evidence dated 24 October 2026

Configuration and setup

How this tool was set up for the run and what the test needed in place. Each row is a fact from the run's records; a fact the records do not hold is left out, not guessed.

Software that produced the output
PyMuPDF4LLM
Build or version
PyMuPDF4LLM 1.28.2
Surface
SDK / library
Set up before the run
A short PDF with styled headings and subheadings over body text had to be converted in one default pass on the full file, with no per-tool flag tuning.
Call
pymupdf4llm.to_markdown('procedure_KAL-SP-06_sample_reception.pdf')
Library version
pymupdf4llm 1.28.2
OCR engine
rapidocr 3.10.0
Tested
24 October 2026 · Vamshidhar Rao Thumula

How this scenario is graded

How we decide Pass, Fail and Not gradable. The same rules apply to every tool tested on this scenario.

How results are decided

Each test case gets one result per tool: Pass, Fail or Not gradable. A test case we haven't run yet shows Untested. There are no partial results.

The rules
  • Pass: the tool did everything the test expected, and nothing it said contradicts the correct answer.
  • Fail: at least one expected behaviour clearly didn't happen; the test case's result says which one and quotes the tool.
  • Not gradable: our evidence couldn't settle the outcome (for example a record we needed is missing). It is never counted as a fail, and the test case's result says what's missing.

Where this sits in the benchmark

This page is one cell of a larger study: one tool, one scenario. Only this benchmark's frame appears here.

History of this result

What has happened to this result since it was first published. Runs and grades are never overwritten: a retest or a re-grade publishes a new result and keeps the earlier one readable.

from the publication record
3 November 2026First publishedConverting a complex PDF into clean Markdown with an open-source library v1

Act on this result

Nothing filed here edits the run or the grade. A challenge opens a review, and a review can produce a new run or a re-grade — which becomes the current result and leaves this one in the history.

This matches what I see

You run the same kind of test against your own setup and get the same behaviour.

Agree →
This does not match

Yours behaves differently. Tell us what you got, with a screenshot if you have one.

Disagree →
Point out an issue

Something here is wrong — a reference value, a transcription, a grade.

Report an issue →
Request a retest

On a newer build, a larger dataset, or your own setup.

Request a retest →
We have fixed this

Tell us what changed and we schedule a rerun of the failing test case. The old result stays as history.

Vendor notice →
Filed against this evidenceNothing yet. Challenges, counter-evidence and fix notices appear here with their outcome, and stay on the page after they are resolved.
Cite this result
aidemos.com/benchmarks/pdf-to-markdown-open-source/results/pymupdf4llm/a-document-with-styled-headings-and-subheadings · 1 pass · 0 fail · coverage 1/1 · graded 2026-10-24

The same record is available as structured data through the AI Demos MCP server, with the counts, the coverage and every per-test-case reason carried as fields.

Verify the proof files

These files support this result. Open a file to inspect the original evidence.

Input · Document (PDF) ↗application/pdf · 43 KB
Output file (Markdown) ↗text/markdown; charset=utf-8 · 7 KB
Proof 1 · Screenshot ↗image/png · 286 KB
File fingerprints (SHA-256)

A fingerprint identifies the exact file used for this result.

Input · Document (PDF)1d53151023b44fca82503e6e748e2a1bed0b2374f54d2dc2d10279009401f34d
Output file (Markdown)8a0cbbb592b848fa0fb85bc96286443f517e278af65087ab6dfdd90a10863a60
Proof 1 · Screenshote9bab24a32e5f1df43b7554ce9c84f5ddcc885e372f181ccbe0119ebdc5acae4