AI Security 6 min read

This PDF MCP server’s redact button only paints a box. My card number stayed.

I ran PaperPilot, a local PDF MCP server with 76 tools, on fake invoices. Its redact tool leaves the card number in the file. Here is the test.

Abstract illustration of a navy page with amber text lines and a flat black rectangle laid on top, with the amber lines still visible underneath and past its edges.

PaperPilot is an open-source PDF MCP server written in Rust. It gives an AI agent 76 tools to merge, compress, extract text, scan for personal data, redact and sign PDFs, and it runs on your own machine over stdio, so your files never go to a hosted API. Version 0.6.1 came out on October 26, and I spent an evening running it on a box with no GPU.

In this post I go through what the server does well, the two tools whose names promise more than they deliver, and a short test you can run before you trust any PDF redaction. My verdict: an honest early project with some good parts, but pdf_redact and pdf_sign should not be handed to an agent that picks its own tools.

Abstract illustration of a navy page with amber text lines and a flat black rectangle laid on top, with the amber lines still visible underneath and past its edges.
A black box drawn on top of text is a picture of a redaction, not a redaction.

Why would a seller want a local PDF MCP server?

An Amazon seller keeps supplier invoices, FBA shipment papers and prep-centre forms in PDFs. They carry names, phone numbers, addresses, sometimes a tax ID or a card on file. Me, I do not love pushing those through somebody’s cloud API just to merge two files, so a server that runs next to the agent sounds right.

Mettons I want an agent to build a supplier packet from three invoices. I made up that scenario, and it is not something inside Trellis.

A stdio MCP server is a program your agent starts and talks to with JSON messages. PaperPilot speaks the MCP specification dated 2025-06-18, and I found it through a Hacker News thread. The repo says no network calls; I did not capture traffic, so I cannot confirm that.

What happened when I ran it on plain Debian?

I downloaded the v0.6.1 Linux x86_64 tarballs from the release page and checked the sha256 against the published files. They matched. Then paperpilot-mcp --version printed 0.1.0, on a release tagged v0.6.1.

Bon. The bigger thing: the binaries did not start on Debian 12. The error was version 'GLIBC_2.43' not found. They are built on GitHub’s ubuntu-latest, so only very new distros work out of the box. I got past it by unpacking Debian sid’s libc6 into /tmp and launching through its loader, which is fine for a test and a bad idea for anything else. I did not try Ubuntu 24.04, so I will not tell you it fails there.

Once running, the handshake was quick: initialize in 5.4 ms and tools/list in 1.6 ms, measured on my own machine over stdio.

How much context do 76 tools cost an agent?

tools/list returned 76 tools and 28,764 JSON characters. At four characters per token that is about 7,200 tokens. That is my estimate, not a tokenizer count. If your client loads every tool definition into every session, you pay it before the first question.

A model choosing among 76 descriptions has 76 chances to pick the wrong one, and a PDF job needs maybe five. My rule for any PDF MCP server, and any big one: expose five tools, not 76. For a seller’s packet that would be pdf_merge, pdf_info, pdf_extract_text, document_pii_scan, and nothing else until a job asks for it.

One friction: pdf_extract_text needs an output path and writes the text to a file, so the text does not come back to the model without a second step.

What worked, and one leak in the PII scan

document_pii_scan did its job. On a synthetic invoice it found a credit card and an email in about 4 ms. On a synthetic FBA sheet it found a social security number and a phone number. It did not flag the name “Jane Doe”, so four of my five planted items were caught. All the data is fake.

The findings come back masked, as ** ** 1111, which is the right idea. The context_snippet next to it still holds the full number: “Card on file: 4111 1111 1111 1111”. A masked value with an unmasked neighbour leaks into the model’s context.

pdf_merge took a 2-page and a 3-page file and pdf_info then reported 5 pages. Boring, correct, under a millisecond.

Does the redact tool actually redact?

No. I ran pdf_redact on a one-page invoice with a box over the card line. It returned PDF redacted successfully. The output was a new PDF with one thing added to the page’s content stream: q 0 0 0 rg 40 696 400 28 re f Q. That is a black rectangle. The original (Card on file: 4111 1111 1111 1111) Tj is still in the file.

I checked three ways. PaperPilot’s own text extraction returns the card number, its own PII scan finds the card again, and plain pypdf, which has nothing to do with this project, finds it too. The CLI paperpilot redact did the same. The ROADMAP lists a “Vector Glyph Redaction Engine” as a future item (6.7.4); I did not find it marked done.

Here is the check I ran myself, on the original invoice and the “redacted” one:

inv1.pdf contains card number: True
inv1_redact2.pdf contains card number: True

Run the same on any redacted PDF, searching for the string you wanted gone:

import pypdf
t = "".join(p.extract_text() for p in pypdf.PdfReader("redacted.pdf").pages)
print("4111 1111 1111 1111" in t)

If it prints True, the redaction failed. Caveat: my test file is a synthetic one-font PDF from a small script, not a scanned invoice, and a scan with no text layer has nothing to leak. A rectangle painted over live text is the failure on any PDF.

Then pdf_sign. Its own description says “Digitally signs a PDF document (Currently a stub).” It returned PDF signed successfully. I grepped the output for /ByteRange and found none, which is all I checked. The description is honest. The result message is wrong, and the model reads the result message.

One more, normal for a stdio server: no path limits. pdf_merge wrote into /tmp/outside/ without asking and pdf_info tried to open /etc/passwd. The agent’s permissions are yours.

How early is this project, and what would I do with it?

Facts from the GitHub API, read on October 30: the repo was created September 28, has 725 stars and 19 forks, and shipped three releases in seven days (October 20, 24 and 26). Of about 241 commits, 214 come from google-labs-jules[bot], an AI coding agent. The LICENSE file contains Apache-2.0 text, while GitHub’s detector says NOASSERTION. The README has a Benchmarks section that says TODO, and its “sub-5ms” claim is the project’s own number, not mine.

That history can mean a fast-moving project with a tireless bot, or thin human review. A redact tool that paints a box and a binary that prints the wrong version lean toward the second, but I cannot tell from outside, so I call this PDF MCP server early. A star is a click, not a code review.

I wrote on October 28 about the gap between “verified” and “trusted” in MCP Skills. Same gap, one level down: this server’s tools answer “success” while the work is not done.

What I would do on a small team:

  • Pin the binary by sha256, or build it yourself on your own base image.
  • Allow-list five tools in the client. Leave redact and sign out.
  • Run it in a container that sees only the files the job needs.
  • Test any redaction with the pypdf check before anyone trusts it.

I did not test OCR, the browser build, the desktop app, the REST gateway, Windows or macOS, large files, form filling, or pdf_validate_signatures. Next I want a real scanned invoice through the OCR path, because that is the file a seller actually has.

Dominic Plouffe

CTO of Trellis, with exits at FuseTalk, 360pi and DeadHeat. I write about making agents and big data work together safely, and I'm building yaaf, an open-source platform for running teams of AI agents that actually finish the job: yaaf.dplouffe.ca