Guides

>

How to Build an AI Pipeline That Creates Product Manuals from a Photo

Outcome: A fully automated pipeline that takes a photo of a product (or just a name) and produces a searchable PDF manual, ingests it into a vector database for voice queries, and links it to a device record on your site — all without manual intervention.

Who This Is For

Home automation enthusiasts running a local AI stack (Home Assistant, Weaviate, local LLMs) who want their product documentation indexed and searchable by voice assistants. You should be comfortable with Docker, REST APIs, and basic Python.

The Problem

Every smart home device arrives with a paper manual that gets lost in a drawer. When you need the factory reset procedure at 11 PM, you’re searching manufacturer websites, Reddit threads, and FCC filings. Multiply that by dozens of devices and it becomes unmanageable.

We built a pipeline that solves this: snap a photo of the product or its manual, and an AI agent identifies the device, finds or writes proper documentation, converts it to a searchable PDF, stores it on the NAS, indexes it in a vector database, and creates a device reference page — all in one pass.

Prerequisites

  • An LLM with vision capabilities — Claude, GPT-4o, or a local multimodal model via LiteLLM. The pipeline uses vision to read product labels and manual pages from photos.
  • Weaviate vector database — for semantic search across ingested manuals. Any vector DB works conceptually, but the pipeline targets Weaviate.
  • Stirling PDF — self-hosted PDF toolkit for HTML-to-PDF conversion and OCR. Runs as a Docker container.
  • NAS or shared storage — a network-accessible folder where PDFs are stored permanently.
  • WordPress with custom post types (optional) — for publishing device reference pages. Any CMS or wiki works as the output target.
  • pdftotext — command-line tool for extracting text from PDFs (part of the poppler-utils package).

Architecture Overview

The pipeline has seven steps. Each step feeds the next, and the whole chain runs from a single trigger — a photo, a product URL, or a text description.

flowchart TD
    INPUT["Photo / URL / Text"] --> IDENTIFY["1. IDENTIFY
Vision AI extracts manufacturer, model, specs"]
    IDENTIFY --> RESEARCH["2. RESEARCH
Web search for official manuals, FCC filings"]
    RESEARCH --> TRIAGE["3. TRIAGE
Score each source: setup, troubleshooting, maintenance?"]
    TRIAGE --> ACQUIRE["4. ACQUIRE PDF
Download official docs or author a guide"]
    ACQUIRE --> INGEST["5. INGEST
Extract text, store in vector DB for semantic search"]
    INGEST --> PUBLISH["6. PUBLISH
Create or update device reference page with links"]
    PUBLISH --> VALIDATE["7. VALIDATE
Confirm: PDF exists, indexed, device page live"]

    style INPUT fill:#fff,stroke:#8b0000,stroke-width:2px,color:#2d2d2d
    style VALIDATE fill:#fdf2f2,stroke:#8b0000,stroke-width:2px,color:#2d2d2d

Step 1 — Identify the Product

The pipeline accepts four input types. Each one resolves to the same structured output: manufacturer, model number, product name, and category.

From a Photo (Most Common)

Send the image to a vision-capable LLM and ask it to extract product details. This works with photos of the device itself, its label/nameplate, the included manual, or the retail box.

The prompt we use:

Look at this image. Extract:
- Manufacturer name
- Model number (exact, including suffixes like .1 or -US)
- Product name / description
- Any visible specs (voltage, wattage, protocol, certifications)
- FCC ID if visible

Return as structured JSON.

If the photo shows an FCC ID, use it to look up additional details:

https://fccid.io/<FCC-ID>

From a Product URL

Fetch the page and extract the product title, brand, model, and specs from the listing. Amazon pages often require a secondary search by ASIN to get the real brand name when the listing title is generic.

From Text

Parse the manufacturer and model directly. If ambiguous, confirm with the user before proceeding.

Step 2 — Research Official Sources

Search for official documentation using targeted queries:

"<manufacturer> <model> user guide PDF site:<manufacturer-domain>"
"<manufacturer> <model> support downloads manual"
"<manufacturer> <model> FCC ID"

Key principle: official sources only. Use third-party sites (ManualsLib, Reddit) to discover what exists, but always trace back to the manufacturer’s domain for the actual download. FCC filings at fccid.io are acceptable as government records.

For manufacturer sites that require JavaScript navigation (common with support portals), use a headless browser like Playwright to navigate and extract direct PDF links.

Step 3 — Triage Before Downloading

This is the step most people skip, and it makes or breaks the pipeline’s usefulness. Not every official manual is actually useful. We learned this the hard way: an FCC filing for a Matter smart power strip contained only a Tuya/WiFi quick-start card for an older product revision — completely wrong protocol, wrong app, useless for setup.

Score every source before downloading:

TRIAGE CHECKLIST:
□ Matches actual product model and variant? (not a different region/generation)
□ Covers setup and commissioning? (step-by-step, not just marketing)
□ Covers troubleshooting? (reset procedure, error meanings, connectivity fixes)
□ Covers maintenance? (care instructions, replacement parts, safety limits)
□ Readable? (coherent English, structured sections, not scan artifacts)

Based on the triage results, choose your acquisition path:

  • Path A — Download: Good official manual passes all checks. Download it directly.
  • Path B — Web Capture: Only web pages exist, no downloadable PDF. Use a headless browser to print the page to PDF.
  • Path C — Author a Guide: No useful official docs exist (or they’re inadequate). Compile information from specs, community threads, and FCC filings into a proper user guide.
  • Path D — OCR a Physical Manual: The user has a paper manual. OCR it to create a searchable PDF.

Always download official docs even if they’re poor — they have reference value. The question is whether you also need to author a guide.

Step 4 — Acquire and Store the PDF

Storage Layout

Use a flat folder structure on your NAS — one folder per product, named by brand and product type:

/mnt/nas_storage/Manuals/
├── Worx WG898 Edger/
│   └── Worx_WG898_OwnersManual.pdf
├── UseeLink Matter Power Strip/
│   ├── UseeLink_MatterPowerStrip_FCC_Manual.pdf
│   └── UseeLink_MatterPowerStrip_UserGuide.pdf
├── Unifi/
│   └── ...
└── Cyberpower/
    └── ...

No nested category subfolders. Keep it simple — you’ll search by content via the vector database, not by browsing folders.

Path A — Direct PDF Download

curl -sL -A "Mozilla/5.0"   -o "/mnt/nas_storage/Manuals/<Product>/<Manufacturer>_<Model>_<DocType>.pdf"   "<manufacturer-pdf-url>"

Verify the download: file exists, non-zero bytes, and pdftotext can extract text from it.

Path B — Web Capture

When only a web page exists, use a headless browser to print it to PDF. Playwright works well for this:

# Using Playwright MCP or similar headless browser
# Navigate to the support page, then print to PDF
# Tag as "_WebCapture" in your metadata so you know the source type

Path C — Author a Guide with AI

This is where the pipeline really shines. When official docs don’t exist or are inadequate, the AI compiles a proper user guide from multiple sources: Amazon listings, FCC filings, community forums, and any photos the user provided.

The authored guide follows a consistent structure:

  1. Product identification (model, specs, certifications)
  2. Setup and commissioning (step-by-step)
  3. Daily use (controls, indicators, behavior)
  4. Troubleshooting (common issues and fixes)
  5. Factory reset procedure
  6. Maintenance and safety
  7. Limitations (what the device cannot do)
  8. Smart home integration notes

Convert the guide to a styled PDF using Stirling PDF’s HTML-to-PDF endpoint:

# Write styled HTML to a temp file (not saved permanently)
# Then convert via Stirling PDF:

curl -s -X POST "http://<stirling-pdf-host>:7890/api/v1/convert/html/pdf"   -F "fileInput=@/tmp/guide.html"   -o "/mnt/nas_storage/Manuals/<Product>/<Manufacturer>_<Model>_UserGuide.pdf"

# Clean up temp file
rm /tmp/guide.html

PDF styling tips: Use a clean sans-serif font (Segoe UI, Calibri, Arial) at 11pt, dark navy headings (#1a1a2e), striped table rows, monospace code blocks on a light gray background, and US Letter page size with 0.75-inch margins. The goal is a professional-looking document that’s easy to read on screen and in print.

Path D — OCR a Physical Manual

For paper manuals, use Stirling PDF’s OCR endpoint (which uses Tesseract under the hood):

curl -s -X POST "http://<stirling-pdf-host>:7890/api/v1/misc/ocr-pdf"   -F "fileInput=@/tmp/scanned_manual.pdf"   -o "/mnt/nas_storage/Manuals/<Product>/<Manufacturer>_<Model>_OCR.pdf"

If OCR quality is poor (common with photographed pages rather than scans), fall back to having the vision LLM read the page content directly and feed it into Path C to produce a clean authored guide.

Step 5 — Ingest into the Vector Database

This step is what makes the manuals useful beyond just being files on a NAS. By ingesting the text into a vector database, every agent and voice assistant in your home can search manuals semantically.

Create the Collection (One-Time Setup)

Create a dedicated collection in Weaviate for home manuals:

curl -s -X POST "http://<weaviate-host>:8080/v1/schema"   -H "Content-Type: application/json"   -d '{
    "class": "HomeManuals",
    "vectorizer": "text2vec-transformers",
    "properties": [
      {"name": "title", "dataType": ["text"]},
      {"name": "content", "dataType": ["text"]},
      {"name": "manufacturer", "dataType": ["text"]},
      {"name": "model", "dataType": ["text"]},
      {"name": "product_name", "dataType": ["text"]},
      {"name": "category", "dataType": ["text"]},
      {"name": "manual_type", "dataType": ["text"]},
      {"name": "file_path", "dataType": ["text"]},
      {"name": "source_url", "dataType": ["text"]},
      {"name": "file_hash", "dataType": ["text"]}
    ]
  }'

Extract Text and Ingest

For each PDF, extract the text and create a Weaviate object:

# Extract text from the PDF
pdftotext "/mnt/nas_storage/Manuals/<Product>/<file>.pdf" -

# Limit to ~8000 characters for the vector embedding
# If extraction fails, use a product summary as the content field

Before ingesting, check for duplicates by file path:

curl -s -X POST "http://<weaviate-host>:8080/v1/graphql"   -H "Content-Type: application/json"   -d '{
    "query": "{ Get { HomeManuals(where: {path: ["file_path"], operator: Equal, valueText: "<path>"}) { title } } }"
  }'

Then ingest:

curl -s -X POST "http://<weaviate-host>:8080/v1/objects"   -H "Content-Type: application/json"   -d '{
    "class": "HomeManuals",
    "properties": {
      "title": "WORX WG898 Owners Manual",
      "content": "<extracted text, max 8000 chars>",
      "manufacturer": "WORX",
      "model": "WG898",
      "product_name": "20V 7-inch Cordless Brushless Edger",
      "category": "Outdoor/Garden",
      "manual_type": "OfficialManual",
      "file_path": "/mnt/nas_storage/Manuals/Worx WG898 Edger/Worx_WG898_OwnersManual.pdf",
      "source_url": "https://www.worx.com/...",
      "file_hash": "<sha256 of file>"
    }
  }'

Verify the Ingestion

Test with a natural language query to confirm semantic search works:

curl -s -X POST "http://<weaviate-host>:8080/v1/graphql"   -H "Content-Type: application/json"   -d '{
    "query": "{ Get { HomeManuals(nearText: {concepts: ["how to reset the power strip"]}, limit: 3) { title manufacturer model file_path } } }"
  }'

You should see results ranked by semantic relevance. “How to reset the power strip” should return the power strip manual first, even if the word “reset” appears in multiple documents.

Step 6 — Publish a Device Reference Page

The final step creates a device reference page in your CMS that links back to the stored manuals and notes that the docs are searchable via the vector store. This gives humans a browsable entry point while agents use the vector search.

If a device page already exists for this product, update it with a “Reference Documents” section. If not, create one — either manually or via an automated device pipeline if you have one.

The reference section should include:

  • Links to the original source URLs
  • NAS file paths for direct access
  • A note that docs are indexed in the vector store for voice/agent queries

Step 7 — Validation Checklist

Do not consider the pipeline run complete until all checks pass:

VALIDATION CHECKLIST:
□ PDF exists at the correct NAS path and is non-zero bytes
□ Source was triaged — you know whether it's official, authored, or captured
□ Weaviate object exists in HomeManuals collection (query by file_path to confirm)
□ Semantic search returns the document for relevant natural language queries
□ Device reference page exists with manual links (if using a CMS)
□ No duplicate Weaviate objects or duplicate device pages
□ No leftover temp files (.md, .html) in the Manuals folder

Lessons Learned Building This

We tested this pipeline with two real products — a UseeLink Matter Smart Power Strip (a Chinese OEM brand with essentially no documentation) and a WORX WG898 Edger (a well-known brand with proper manuals). Here’s what we learned:

  • Triage is essential. The FCC manual for the power strip was a WiFi/Tuya quick-start card for an older product revision. Without triage, we’d have ingested a useless document and called it done. Instead, we caught it and authored a proper Matter-focused guide.
  • AI-authored guides can be better than official docs. For the power strip, no official Matter setup guide existed. The AI compiled information from FCC filings, Amazon specs, community forums, and protocol documentation into a structured guide that actually covered setup, troubleshooting, and Home Assistant integration — more useful than any single source.
  • Well-documented products are fast. The WORX edger had a proper PDF on worx.com. The pipeline downloaded it, verified it passed triage, ingested it, and was done in under a minute. No authoring needed.
  • Flat folder structure wins. We tried category subfolders first and it added complexity with zero benefit — you search by content via the vector database, not by browsing folders.
  • No intermediate files. Early runs left .md and .html source files in the Manuals folder alongside the PDFs. Only PDFs belong there. Generate intermediates in /tmp and clean up.
  • Stirling PDF is the right tool for HTML-to-PDF. We tried multiple approaches. Stirling PDF (self-hosted, Docker) handles HTML-to-PDF conversion cleanly and also provides OCR when you need to process scanned documents.

Making It Accessible to Voice Assistants

Once manuals are in Weaviate, any agent with HTTP access can query them. Voice assistants can answer questions like:

  • “What’s the reset procedure for the power strip?”
  • “What’s the max wattage on the Cyberpower UPS?”
  • “How do I replace the edger blade?”

The semantic search handles natural language variations — “how to factory reset” and “restore to defaults” both find the same content. This is the real payoff: documentation that’s not just stored but genuinely accessible when you need it.

Troubleshooting

  • Manufacturer site requires login: Record the URL, skip the download, and ingest metadata only. Note in the device page that the manual is “available at [url]” without a local copy.
  • pdftotext returns empty output: The PDF may be image-only (scanned). Use Stirling PDF’s OCR endpoint first, then re-extract. If still empty, use a product summary as the Weaviate content field.
  • Stirling PDF HTML-to-PDF fails: Fall back to a headless browser (Playwright) print-to-PDF. The styling won’t be as clean but the content will be correct.
  • Weaviate returns no results for a query: Check that the content field was populated (not empty). Re-extract text and update the object. Also verify the vectorizer module is running.
  • Wrong product variant in the manual: This is a triage failure. Re-run Step 3 with stricter model matching. Many manufacturers reuse manual PDFs across product generations — always check the protocol version and app references match your actual device.

What’s Next

Once the pipeline is working, the natural extensions are:

  • Trigger from Discord or chat: Wrap the pipeline as a skill that an AI agent can invoke from a chat message — “create a manual from this picture” with an attached photo.
  • Batch processing: Walk through your home and photograph every device label. Run them all through the pipeline in sequence.
  • Automatic updates: Periodically re-check manufacturer URLs for updated manuals and re-ingest when changes are detected.
  • Cross-device queries: “Which of my devices support Matter?” — answerable once all manuals are indexed with proper metadata.

Categories:

|

Domains:

|

Use Cases:

|

Tags: