Testing 5 Open-Source PDF Parsing Tools

There are many PDF parsing tools. Once they are wired into RAG systems, enterprise knowledge bases, or document intelligence pipelines, the question is rarely just “can it OCR?” The output has to stay useful: stable text, tolerable reading order, tables and formulas that survive well enough, and content that can still be chunked, indexed, and cited.

I tested 5 open-source PDF/document parsing tools on the same public dataset, the same server, and the same lightweight metrics. I am not affiliated with these projects, and the conclusions do not rely on private samples.

Tools

Tool	Type	Evaluation Role
Docling	Document conversion / engineering baseline	Engineering usability, speed, and Markdown output
MinerU	End-to-end PDF parser	A full parsing pipeline from the OpenDataLab ecosystem
PaddleOCR-VL-1.6	Multimodal document parser	PaddleOCR’s current VLM document parsing route
DeepSeek-OCR 2	VLM/OCR	A newer visual OCR and document understanding route
dots.mocr	Multimodal OCR/parser	A newer bbox/category/text structured-output route

pdftotext is not in the main table. It is useful as a diagnostic baseline for digital PDFs, but this test focuses on page-image parsing: newspapers, textbooks, slides, scanned pages, tables, and formulas.

Dataset

The main dataset is opendatalab/OmniDocBench. It covers papers, textbooks, slides, newspapers, scanned pages, and other document types, which makes it suitable for a public PDF parsing comparison that other people can inspect.

The main table uses one shared article-300 manifest:

Item	Setting
Dataset	OmniDocBench
Sample size	300 pages
Input	Page images
Task	Parse document pages into text, Markdown, or structured content
Goal	Compare runnability, speed, failure rate, and text similarity

The main conclusions only use the same completed 5-tool run. I do not mix different sample sizes, different execution modes, or partially completed results.

Testbed

All tools ran on the same test server. The performance numbers are relative results in this environment, not universal deployment claims.

Item	Configuration
CPU	2 x Intel Xeon Silver 4314 @ 2.40GHz
CPU cores/threads	32 physical cores / 64 logical threads
Memory	512 GiB
GPU	2 x NVIDIA GeForce RTX 3090
GPU memory	24 GiB x 2
NVIDIA Driver	550.144.03
CUDA	12.4

Metrics

The main table only includes metrics that are fairly comparable across all 5 tools:

Metric	Meaning
Success rate	Whether the tool completed page processing and produced output
Compute	CPU/GPU execution mode
Run mode	Batch, per-page adapter, or API service
Mean seconds/page	Total wall time divided by page count
Median seconds/page	Less affected by a small number of very slow pages
CER	Character error rate, lower is better
WER	Word error rate, lower is better

CER/WER do not measure full PDF parsing quality. The tools produce very different formats: Markdown, plain text, or JSON blocks. Aligning all of them directly against ordered ground-truth text counts Markdown markers, table formatting, and reading-order differences as errors. Here, CER/WER are text-similarity references, not the only ranking basis.

Layout F1, Table F1, and Formula Exact Match are not included in the main leaderboard. The reason is that the 5 tools expose structure through different sources: Docling uses prov.bbox, MinerU/PaddleOCR-VL/dots.mocr use their own JSON blocks, and DeepSeek-OCR 2 uses det markers inside Markdown. These metrics are useful for structural diagnostics, but they should not replace the main leaderboard.

To make the benchmark richer than CER/WER alone, this article also records two groups of extended metrics:

Extended Metric	Purpose	Main Table
Text coverage	Whether the tool reliably produces text that can be processed further	Yes, engineering reference
Blocks coverage	Whether the tool has a basis for structured output	Yes, engineering reference
JSON validity	Whether structured output can be parsed reliably	Yes, adapter reference
Table similarity	Compare extracted table content with annotations	No, diagnostic only
Formula similarity	Compare extracted formula content with annotations	No, diagnostic only
Layout F1 / Table Layout F1 / Formula Layout F1	Evaluate bbox/category structure recovery	No, diagnostic subset

These extended metrics expose output shape and adapter maturity. They do not replace the main table, and they should not be used to claim that one model is universally better. A stricter structure benchmark still needs all 5 tools converted into equivalent blocks.json, table, and formula schemas.

Results

All 5 tools completed the same OmniDocBench article-300 run.

Tool	Compute	Run mode	Success	Mean CER	Median CER	Mean WER	Median WER	Mean s/page	Median s/page	Pages/min	Total time
Docling	CPU	per-page adapter	300/300	0.534699	0.503003	2.384433	0.984725	5.97	3.99	10.05	29.8 min
MinerU	GPU/CPU pipeline	batch, amortized	300/300	0.526917	0.309285	1.728110	0.875000	0.76	0.76	78.62	3.8 min
PaddleOCR-VL-1.6	GPU	per-page HF/adapter	300/300	0.859234	0.379661	3.225547	0.871582	25.67	15.62	2.34	128.4 min
DeepSeek-OCR 2	GPU	per-page API	300/300	5.869590	0.833291	7.184052	1.056094	55.32	28.84	1.08	276.6 min
dots.mocr	GPU	per-page HF/adapter	300/300	1.197031	0.652942	2.971242	1.911991	68.17	49.38	0.88	340.8 min

Mean is the average and can be pulled upward by a small number of difficult pages. Median is the middle value and better reflects typical-page behavior. MinerU’s runtime comes from a batch runner, so its 0.76 seconds/page number is an amortized batch result and should not be treated as the same thing as per-page online latency.

Extended Metrics: Output Stability

The following table covers all 5 tools and reflects whether their outputs are stable and programmatically usable.

Tool	Text Output	Blocks Output	Valid JSON	Note
Docling	300/300	300/300	300/300	Exports `prov.bbox` via `export_to_dict()`
MinerU	300/300	300/300	300/300	Can enter structure metric computation
PaddleOCR-VL-1.6	300/300	300/300	300/300	Can enter structure metric computation
DeepSeek-OCR 2	300/300	300/300	300/300	Blocks exported from Markdown det markers
dots.mocr	300/300	300/300	295/300	Structurally useful, but JSON stability needs attention

This table is closer to engineering reality than CER/WER alone. RAG preprocessing often needs more than text: page references, block boundaries, heading hierarchy, table regions, and traceability. In this adapter setup, all 5 tools can produce structured blocks, but the source and stability differ. dots.mocr has 5 pages that cannot be parsed as JSON directly; DeepSeek-OCR 2 blocks are extracted from detection markers embedded in Markdown, which is noisier than a native JSON schema.

Extended Metrics: Structure Diagnostics

These structure metrics are diagnostic only and are not the 5-tool main leaderboard. Docling was missing these values in the first adapter because only Markdown/text was saved; after adding export_to_dict(), its prov.bbox fields can be converted into normalized blocks. DeepSeek-OCR 2 can also export bbox data from Markdown ref/det markers, but that path is more sensitive to output-format noise than native JSON blocks.

Tool	Layout F1	Table F1	Formula F1	Table Similarity	Formula Similarity	Basis
Docling	0.496621	0.876106	0.453172	0.189346	0.001937	Normalized blocks from Docling `prov.bbox`
MinerU	0.500484	0.914980	0.626466	0.452320	0.283428	Normalized block JSON
PaddleOCR-VL-1.6	0.707293	0.970954	0.000000	0.913666	0.598309	Formula layout labels did not match normalized formula blocks
DeepSeek-OCR 2	0.133424	0.560748	0.126354	0.286046	0.376911	bbox extracted from Markdown det markers
dots.mocr	0.605774	0.939271	0.587065	0.840033	0.611329	bbox/category/text JSON; 5 JSON pages need repair

Several things stand out. Docling has a high Table F1, so it can locate table regions reasonably well, but its table-content similarity is low. Finding a table and reconstructing its content are different problems. PaddleOCR-VL-1.6 and dots.mocr are stronger on table-content similarity, so table-heavy documents deserve a separate test. DeepSeek-OCR 2 det markers are useful for structural diagnostics, but the low overall Layout F1 suggests that bbox/category output needs stricter prompting and parsing constraints. PaddleOCR-VL-1.6’s Formula F1 of 0 also does not mean formula content is entirely unusable; it means the current normalization did not reliably match its formula layout labels.

The structure results should guide the next round of evaluation, not act as the final ranking.

Observation 1: Docling Is a Strong Engineering Baseline

Docling is a strong CPU engineering baseline in this run, with a mean of 5.97 seconds/page and a median of 3.99 seconds/page. Installation, invocation, and output are straightforward, which makes it useful for measuring the lower bound of fast Markdown/text conversion.

If the job is to convert many documents quickly into readable Markdown or text, Docling deserves early validation. After adding the structure adapter, it can also export bbox/category blocks and reaches a Layout F1 of 0.496621 on article-300. For RAG systems that are sensitive to table content, formula content, and figure-caption relationships, manual sampling is still necessary.

Observation 2: MinerU Has the Strongest Text Metrics and Throughput

MinerU has the lowest Mean CER, the lowest Mean WER, and the highest throughput in this article-300 run. It uses a batch runner, so the 0.76 seconds/page number is amortized batch throughput rather than per-page online latency.

On lightweight text metrics:

Dimension	Best in This Run
Mean CER	MinerU
Median CER	MinerU
Mean WER	MinerU
Median WER	PaddleOCR-VL-1.6

This does not mean MinerU wins every dimension. On these 300 public pages and under this normalization method, MinerU has stronger ordered-text similarity and batch throughput, making it a primary candidate for RAG preprocessing.

Observation 3: PaddleOCR-VL Is Stable; DeepSeek-OCR 2 and dots.mocr Depend More on Output Normalization

PaddleOCR-VL-1.6 has higher Mean CER/WER than MinerU and Docling, but its Median CER/WER is close to MinerU. That suggests it is not weak on typical pages; a subset of complex pages pulls up the average.

DeepSeek-OCR 2 and dots.mocr both completed 300/300 pages, but their CER/WER values are clearly higher. DeepSeek-OCR 2’s Mean CER is much higher than its Median CER, which means a small number of pages diverged heavily from ordered ground truth and raised the average.

This should not be reduced to “the model is bad.” Output shape matters a lot:

DeepSeek-OCR 2’s current API prompt and output normalization are still coarse, and formatting differences amplify CER/WER.
dots.mocr outputs bbox/category/text JSON. Comparing JSON-style content directly against ordered text counts structural symbols as text errors.

dots.mocr’s value is mainly in structured output, not pure-text CER/WER ranking. Systems that depend on bbox, category, table HTML, or formula LaTeX should evaluate it through structure metrics; until schemas are fully unified, structure scores should remain diagnostic.

Observation 4: Structured Capability First Reflects Adapter Maturity

The extended metrics show that structured capability is not just about model names. Output protocol stability matters. Docling, MinerU, PaddleOCR-VL-1.6, and DeepSeek-OCR 2 all produce 300/300 structured blocks in this setup. dots.mocr also produces 300/300 block outputs, but only 295/300 can be parsed as JSON directly. It has structural value, but it also needs JSON repair or adapter hardening.

DeepSeek-OCR 2 blocks come from detection markers embedded in Markdown, not from an independent JSON schema. That means diagnostic metrics can be computed, but the result should not be over-interpreted. If a production system needs traceable chunks, page coordinates, table regions, and formula regions, Docling, MinerU, PaddleOCR-VL-1.6, and dots.mocr are more direct structured-output candidates. DeepSeek-OCR 2 needs stricter structured prompts and adapters before it is suitable for production-grade structure evaluation.

Why Structure Metrics Are Separate

The hard part of PDF parsing is structure recovery: reading order, layout blocks, tables, formulas, figures, and captions. Ideally, a benchmark should include:

Metric	Required Shared Output
Layout F1	bbox + category
Table F1 / TEDS	table bbox, cells, or HTML
Formula Exact Match	formula region + LaTeX
Reading order	block order or sortable structure

The 5 tools do not naturally produce the same output schema. To avoid unfair comparisons, the main table does not use structure metrics that are strongly adapter-dependent. A more rigorous structure benchmark needs a unified blocks.json schema and equal-granularity adapters for Docling, DeepSeek-OCR 2, dots.mocr, MinerU, and PaddleOCR-VL. Before that, structure metrics are useful as diagnostics but not as the formal ranking basis.

Selection Guidance

Need	Recommendation
Fast Markdown/text conversion	Start with Docling
General PDF parsing and RAG preprocessing	Start with MinerU, then compare PaddleOCR-VL-1.6
New VLM/OCR exploration	Include DeepSeek-OCR 2 and dots.mocr in the experiment pool
Structured block output	Validate Docling, MinerU, PaddleOCR-VL-1.6, and dots.mocr first
Table content fidelity	Re-test PaddleOCR-VL-1.6 and add manual failure cases
Strong table/formula requirements	Do not rely on CER/WER; run structure metrics and manual failure review
Production deployment	Prioritize failure rate, output stability, throughput, license, and deployment cost

If only two tools move into production-oriented validation, I would start with MinerU and PaddleOCR-VL-1.6. If a CPU speed baseline is needed, keep Docling. If structured multimodal parsing matters, add dots.mocr. DeepSeek-OCR 2 is better treated as a VLM/OCR research candidate in the current setup.

Conclusion

There is no single winner in this run.

All 5 tools completed the public OmniDocBench article-300 run with zero page-level failures.
Docling is the most direct CPU engineering baseline.
MinerU is strongest in the current text metrics and batch throughput, making it a primary RAG preprocessing candidate.
PaddleOCR-VL-1.6 is close to MinerU on typical pages, but complex pages raise its mean errors.
DeepSeek-OCR 2 and dots.mocr are worth tracking, but output normalization and structure stability require more care.
The extended metrics suggest Docling, MinerU, PaddleOCR-VL-1.6, and dots.mocr are more direct candidates for structured block workflows.
To evaluate tables, formulas, and layout rigorously, all 5 tools still need to be normalized into the same structure schema before a Layout/Table/Formula leaderboard should be treated as final.

References

OmniDocBench: https://github.com/opendatalab/OmniDocBench
Docling: https://github.com/docling-project/docling
MinerU: https://github.com/opendatalab/MinerU
PaddleOCR: https://github.com/PaddlePaddle/PaddleOCR
DeepSeek-OCR: https://huggingface.co/deepseek-ai/DeepSeek-OCR-2
dots.mocr: https://huggingface.co/rednote-hilab/dots.mocr