PDF Quality Factory

PDF Quality Factory

A QA loop that turns AI-generated PDFs from "it compiles" into "ready to sell". Validated through repeated blind bake-offs: final outputs scored 8.4/10 (cover) and 9.5/10 (infographic) under blind human-model judging. The core insight: never trust generated code or the claimed output — prove both, every round.

The loop (run ALWAYS, no exceptions)

R0. BRIEF     -> prompt with technical spec + EXPLICIT DESIGN DIRECTION
R1. GENERATE  -> one model generates the PDF-building Python (ReportLab)
R2. PROVE     -> py_compile + run + page count + text extraction checks;
                 auto-fix up to 2 rounds
R3. SEE       -> render pages to PNG and inspect EVERY page (not just the
                 cover): collisions, truncation, hierarchy, fact-check
R4. JUDGE     -> BLIND judge (a different model or fresh context, not told
                 who generated what): 3 strengths + 3 defects + per-criterion
                 scores
R5. FIX       -> specific critique returns to the generator as an explicit
                 "CRITICAL DESIGN DIRECTION" (e.g. "line X truncated",
                 "labels removed", "border 2pt", "thumbnails +15%")
R6. REPEAT    -> R3-R5 until EVERY page scores >= 8/10 and zero functional
                 defects (max 3 extra rounds; typically 2-3 total)
R7. SHIP      -> only now is the product READY TO SELL

Acceptance criteria (no product ships without all of these)

• [ ] 0 text collisions, 0 truncations, 0 placeholders (%d/%s/{}) in the rendered output
• [ ] Fact-check ALL factual content (temperatures, conversions, dates, numbers) against a reliable source
• [ ] Typography: premium embedded fonts (SIL OFL licensed), registered explicitly and embedded in the file (verify with pdffonts: emb=yes). A validated pairing: a grotesque (e.g. Bricolage Grotesque) for titles/prose + a mono (e.g. DM Mono) for data/labels. One generic system font (Helvetica/DejaVu) as fallback only. Cards in a pair get equal heights; bar charts anchored to a common baseline.
• [ ] Cover shows a preview strip of the inside pages (proof of value on the cover itself)
• [ ] Judge score >= 8 on EVERY criterion for the main pages
• [ ] Mental test: "would a buyer pay $6.99 for this and leave 5 stars?"

Brief rules (R0)

• Explicit design direction beats hoping: state palette (2-3 colors), whitespace expectations, font roles, and layout structure in the prompt.
• Ban literal placeholder tokens ("no %d literals") in the prompt itself.
• Require verified facts in the prompt, with the source named.
• Ask for one page per topic; forbid filler repetition.

Generation pitfalls (each one cost a bake-off round)

• Low max_tokens + reasoning models: the model's thinking eats the budget and content comes back empty. Budget generously.
• Unlimited reasoning ("max" effort): the model over-deliberates and hallucinates APIs (e.g. Canvas.setCharSpace), crashing at runtime. A high-but-bounded reasoning setting is the sweet spot.
• Code-gen models are VARIABLE: roughly 1 in 3 rounds throws a trivial syntax error. Always py_compile before running.
• A second model used as BLIND judge (not the generator) catches defects the generator's own self-review misses. Use it; don't self-grade.
• Rename the winning script after verification and reuse it as the base for the next product — don't regenerate from scratch.

Dual-engine cover technique

When ReportLab art limits the cover: generate the cover as HTML/CSS (gradient, large typography, badge) and render it with headless Chromium (page.pdf(..., prefer_css_page_size=True), HTML with @page{size:A4;margin:0}, body 210x297mm). Keep the body in ReportLab; merge with pypdf and add bookmarks (Cover / Introduction / chapters). Rules: the rest of the pipeline stays unchanged; in the HTML cover use @font-face pointing at your font files (never fall back to system fonts) and confirm pdffonts shows emb=yes on the final PDF.

Tooling requirements

• reportlab, pypdf for generation/merging
• poppler-utils (pdfinfo, pdftotext, pdftoppm, pdffonts) for verification and rendering
• An image-capable model (or human eyes) for R3/R4 page-by-page review
• Fonts: download any SIL OFL fonts (Google Fonts) and embed them

Output format

The loop's final report states, per page: score, defects found and fixed, fonts embedded (yes/no), placeholder scan (clean/dirty), fact-check results, and the final verdict READY TO SELL or NOT SHIPPED (with reason).