Converting PDF to Markdown for LLMs and RAG

If you are building a retrieval pipeline over a pile of PDFs, the extraction step decides the quality of everything downstream. Feed in flat text and your chunker has to guess where one topic ends and the next begins. Feed in Markdown and the document tells it.

Plain text extraction throws away the one signal that is most useful for chunking: hierarchy. A 40-page manual becomes an undifferentiated wall of prose. Your splitter falls back to counting characters, which means chunks routinely start mid-sentence and straddle two unrelated sections. Retrieval then returns a chunk that is half about billing and half about shipping, and the model has to work out which half answers the question.

Markdown keeps the headings. That gives a splitter real boundaries to cut on, and it lets you attach the heading path to each chunk as metadata. "Refund Policy → Partial refunds" in front of a chunk is a large amount of context for a very small number of tokens.

PDFs do not mark headings. There is no h1 in the file, only text drawn at various sizes. So the converter measures instead: it collects every font size on every page, finds the most common one — that is body text — and treats sizes meaningfully larger than the body as headings, ranked so the largest becomes h1 and the next becomes h2.

This works well on documents with consistent typography, which covers most reports, papers, manuals, and books. It works less well on marketing material where text size varies for decoration rather than structure. The result metadata tells you how many heading levels were detected, which is a quick sanity check: a 60-page manual reporting one level means the structure was not recovered and you should inspect the output before indexing it.

Lists survive as lists, which matters when the answer to a question is a set of steps. Emphasis is preserved where it can be detected. And because Markdown is plain text, it costs almost nothing in tokens compared to HTML, where every tag is billable overhead in a context window.

That last point is worth being concrete about. The same document as HTML can be substantially more tokens than as Markdown, and none of the extra tokens carry meaning the model needs. When you are packing retrieved context into a limited window, that difference is real.

The documents people want to search are usually the ones they cannot casually upload: internal policies, contracts, customer records, unpublished research. Most PDF-to-Markdown services process server-side, so building your pipeline on one means every document passes through infrastructure you do not control.

This converter runs in the browser. For a one-off batch during development, that is often enough on its own. For production ingestion you will want a server-side equivalent, but the same logic applies: parse locally, index locally.

Before writing any ingestion code, convert two or three representative files and read the Markdown. Checking whether the heading levels match the real structure takes a minute and saves debugging a retrieval problem that was actually an extraction problem.