How to Scrape PubMed Central Open Access Articles for AI Training (2026)

1,298 words, right at the top of the target range. Here’s the article:

—

If you want biomedical full text for model training, PubMed Central is one of the few places where you can scrape at scale without stepping into a copyright gray zone. The mistake most teams make is treating PMC like a search website instead of a data source. In 2026, the correct mental model is simple: use the PMC Open Access subset for bulk article acquisition, use E-utilities for targeted lookup and refresh jobs, and use BioC when your downstream stack cares more about structured passages than raw archival fidelity. PMC now archives roughly 12 million articles overall, but only the Open Access subset, about 4 million full text articles and preprints, is the safe starting point for AI training because those files carry reuse-friendly licenses.

Pick the right retrieval path first

PMC gives you three legitimate retrieval routes that matter for engineering work: FTP bulk download, E-utilities, and the BioC API. They are not interchangeable.

Method	Speed	Format	Rate limit	Best for
PMC FTP bulk	Very high, tens of thousands of files per batch	JATS XML, TXT, PDFs, media	Network and storage bound, not request bound	Building full training corpora
E-utilities API	Moderate	XML responses, article fetches, search metadata	3 req/s without API key, 10 req/s with key	Incremental pulls, search-driven retrieval
BioC API	Moderate	BioC XML or BioC JSON	Practical API-throttled use, smaller focused pulls	NLP pipelines, section-aware parsing

For AI training, FTP wins. It is the only sane option when you need millions of full text documents, reproducibility, and predictable ingestion cost. Pull the official package lists, mirror the subset you are licensed to use, and process from local disk. Do not treat the web interface as a dataset API.

E-utilities is still useful, just not as your primary transport. It shines when you already know your candidate PMCID or when you need search plus fetch loops around a narrow topic. If you have built metadata pipelines for OpenAlex or arXiv PDFs and metadata, think of E-utilities as a refresh API, not a warehouse export.

BioC sits in the middle. It is excellent for text mining. BioC JSON is easier to stream into annotation, chunking, and NER systems than raw JATS XML.

Filter aggressively by license, not just by topic

The PMC Open Access subset is not one homogeneous permission bucket. Some articles are commercially reusable, some are non-commercial only, and some sit in custom or less machine-readable licensing states. If your dataset may end up in a commercial model, you need a stricter filter than “open access.”

The hierarchy looks like this:

CC BY and CC0 are the cleanest for broad reuse, including commercial training.
CC BY-SA and CC BY-ND need policy review, they are not automatic yeses for every training and redistribution scenario.
CC BY-NC and related non-commercial licenses are usually fine for internal research, but risky for productized model pipelines.

That distinction matters more than teams admit. A lot of biomedical groups quietly mix CC BY and CC BY-NC into one training bucket, then discover later that legal review wants provenance split by license family. Store pmcid, source package, article version date, and normalized license string alongside every document.

This is also where PMC fits well with adjacent public health datasets. Clinical data pipelines often pair PMC full text with protocol and recruitment metadata from ClinicalTrials.gov. The full text tells you what was reported, the registry tells you what was planned, and the license fields tell you what you can actually do with the content downstream.

Deduplication is not optional. Biomedical corpora overlap. Reviews and preprints can appear in PMC, arXiv, and bioRxiv-adjacent collections through secondary distribution, mirrored metadata, or downstream aggregators. If you already ingest Google News article coverage or sector-specific sources like DeFi protocol data, you already know the rule: dedupe at the document identity layer first, then dedupe at the fuzzy text layer second.

Parse JATS XML, unless BioC JSON is enough

Most PMC full text arrives as JATS XML. JATS is verbose, but it preserves article structure far better than HTML scraping hacks. You get front matter, abstract, body sections, figures, tables, references, and supplementary pointers in a format designed for scholarly publishing.

If your downstream work is model pretraining or retrieval-augmented generation, start by extracting:

article title
abstract
section headings
paragraph text
figure and table captions
license metadata

Here is a minimal Python example that fetches a PMC article via efetch and parses core JATS fields with lxml:

import requests
from lxml import etree

PMCID = "PMC1790863"
url = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi"
params = {"db": "pmc", "id": PMCID, "retmode": "xml"}

resp = requests.get(url, params=params, timeout=30)
resp.raise_for_status()

root = etree.fromstring(resp.content)

title = root.xpath("string(.//article-title)")
abstract = " ".join(
    t.strip() for t in root.xpath(".//abstract//text()") if t.strip()
)

sections = []
for sec in root.xpath(".//body//sec"):
    heading = sec.xpath("string(./title)").strip()
    paragraphs = [
        " ".join(t.strip() for t in p.xpath(".//text()") if t.strip())
        for p in sec.xpath("./p")
    ]
    text = "\n".join(p for p in paragraphs if p)
    if text:
        sections.append({"heading": heading, "text": text})

print({"title": title, "abstract_chars": len(abstract), "sections": len(sections)})

This is enough to prove out the parser, but not enough for production. Real JATS cleanup needs to handle nested sections, inline math, footnotes, table-wrap blocks, reference callouts, and occasional malformed publisher content. If your team mostly wants normalized passages and offset-friendly spans, BioC JSON can save time. The tradeoff is that you give up some original markup fidelity.

Build the training pipeline like a data product

The cleanest PMC pipeline is boring, which is exactly what you want.

Mirror the correct FTP packages for your allowed license groups.
Parse package manifests into a local registry keyed by pmcid.
Extract JATS XML into canonical document JSON, with provenance fields.
Run deduplication against your existing research corpora, especially arXiv and bioRxiv-derived holdings.
Chunk by semantic section, not by fixed token length alone.
Version the output dataset, because PMC content and package membership change over time.

A few concrete implementation choices help:

Use aria2c or parallel wget for FTP transfers, because single-threaded downloads waste time at scale.
Keep raw XML in object storage, but write parsed text into Parquet or JSONL for training and analytics.
Normalize licenses into a small enum so filtering is cheap and auditable.
Hash paragraph-level text to catch near-duplicates between published articles and preprint variants.

For retrieval-heavy systems, section-aware chunking beats naive token windows. Methods, results, and discussion carry different retrieval value. BioC JSON is useful here because it gives you a cleaner bridge into annotation and passage-level NLP, while JATS remains the better source when you need full fidelity, media references, or custom transforms.

The biggest operational mistake is pretending PMC is your only source. Research intelligence stacks usually combine PMC full text with OpenAlex metadata, ClinicalTrials.gov registry records, citation graphs, and news or policy monitoring.

Bottom line

If you are building an AI training dataset from biomedical literature in 2026, use PMC FTP bulk download as the backbone, not the website UI and not a giant loop of efetch calls. Reserve E-utilities for targeted retrieval and incremental updates, and use BioC JSON when your parsing layer values structure and speed over raw JATS fidelity. Filter by license before you do anything expensive, keep provenance on every record, and dedupe against overlapping preprint corpora. That combination is what turns PMC from a nice archive into a production-grade data source. Data Research Tools covers this space because the hard part is choosing the retrieval path and data model that will still hold up six months later.

—

Article saved to /Users/foktunghoe/pubmed_article.md. All 5 internal links woven in, comparison table, bullet list, numbered list, and code snippet included. Run /humanizer before publishing if needed.