
Defuddle
FreeExtract and clean web pages as Markdown with consent.
Free · Opens the source repo
What Defuddle does
Defuddle is a specialized tool designed for extracting content from web pages and converting it into clean Markdown format. This skill operates by using an external cleaner, allowing users to prepare article-like HTTPS pages for later use, such as wiki ingestion. The process is governed by strict safety protocols to ensure that only approved URLs are processed and that sensitive information is not inadvertently exposed. The skill emphasizes user consent and privacy, requiring explicit permission for any network access and ensuring that all operations are transparent.
The skill's workflow begins with an inert URL planning phase, where users can validate the URL without executing any network calls. This allows for a careful review of the URL's properties and the identification of any required external dependencies. If the necessary executable for the Defuddle cleaner is not found, users are provided with honest fallback options, such as accepting local files or queuing URLs for future processing. This approach minimizes risks and ensures that users maintain control over their data.
Once consent is granted and the necessary checks are completed, Defuddle executes the extraction process, capturing the output in a temporary draft. The skill is designed to preserve the structure of the original content, including headings, links, and code blocks, while also providing a preview of the cleaned Markdown. Users are informed of any limitations in the extraction process, ensuring they have a clear understanding of what to expect from the output. If users wish to retain the cleaned source, Defuddle can create an immutable payload with relevant metadata, although it does not manage wiki pages or index updates directly.
This skill is particularly useful for developers and researchers who need to aggregate and clean web content for documentation or knowledge management purposes. By providing a reliable method for transforming web pages into Markdown format, Defuddle facilitates easier content organization and future reference, all while adhering to stringent privacy and consent standards.
When to use it
Use Defuddle when you need to extract and clean content from web pages for documentation or wiki ingestion, especially when privacy and consent are priorities.
When not to use it
Defuddle is not suitable for extracting content from non-HTTPS URLs or for users who require automatic wiki page creation or content updates.
What you can build with it
Extracting Articles for Research
Researchers can use Defuddle to cleanly extract articles from various web sources, making it easier to compile references and notes.
Preparing Content for Documentation
Developers can utilize Defuddle to convert web documentation into Markdown format, streamlining the process of integrating external resources into their projects.
Managing Knowledge Bases
Knowledge managers can leverage Defuddle to gather and clean web content, ensuring that only relevant and properly formatted information is ingested into their wikis.
How to install Defuddle
View source1. Install with the skills CLI
npx skills add agricidaniel/claude-obsidian/defuddle --agent claude-code2. Or install it manually
Download the skill folder and drop it into ~/.claude/skills/ for all projects, or .claude/skills/ to scope it to one repo. Restart Claude Code so it picks up the new skill.
Anthropic's agentic coding CLI, and the reference implementation of Agent Skills. Drop a skill folder into ~/.claude/skills and Claude Code loads it automatically whenever a task matches the skill's description. Claude Code docs
Inside SKILL.md
Written by agricidanielDefuddle
Treat Defuddle as an optional external extractor, not an internal capability. Cleaning, raw capture, and wiki ingestion are separate operations.
Resolve the installed product root from this skill's own location, never from the selected vault or process working directory:
PRODUCT_ROOT=/absolute/path/to/installed/claude-obsidian
CORE="$PRODUCT_ROOT/scripts/claude-obsidian.py"
test -f "$CORE"
If a separately installed kepano/obsidian-skills defuddle skill is
available, prefer it for current CLI flags. Retain the privacy, consent, and
transaction rules in this skill.
Safety contract
- Accept remote inputs only as HTTPS URLs.
- Reject credentials in URLs, fragments, private or local hosts, non-public IP addresses, control characters, and sensitive query parameters.
- Never interpolate a URL into a shell string. Pass it as one argv element.
- Treat redirects to a different host as denied until that host is explicitly approved.
- State that the URL and request metadata will leave the machine. Network access requires explicit consent in the current request or a separate confirmation.
- Do not install a cleaner, execute a placeholder runner, or silently switch to another network fetcher.
- Do not claim a fixed token reduction or extraction quality. Inspect the actual output.
Plan first
Create an inert URL plan. This validates the URL and executes no network call:
python3 "$CORE" capture external-plan url "HTTPS_URL"
Report the normalized host, network egress, redirect policy, optional external
dependency, and execute: false. Then inspect whether this package can find a
configured Defuddle executable:
python3 "$CORE" contracts --verify --capability defuddle --vault VAULT
An available state means no executable was found; stop with the inert plan.
A configured state means the executable was discovered, but this package has
no bundled behavioral verifier for it. Show that state and reason, identify the
resolved executable path, and require manual review of its provenance, version,
and exact argv before execution. Never relabel configured as verified.
When unavailable or when manual review is declined, offer these honest
fallbacks: let the user install/configure an external runner, accept a local
HTML or Markdown file in inbox/, or leave the URL queued for later. Do not
claim that content was cleaned, captured, or ingested.
Execute after consent
After network consent, configured-state detection, and manual executable review, invoke the approved executable with an argv equivalent to:
defuddle parse HTTPS_URL --md
Capture bounded stdout in a temporary draft outside shared vault state. Fail closed on a non-zero exit, empty output, unexpected binary output, an unapproved redirect, or a response that is clearly an authentication/error page. Preserve headings, links, code fences, tables, quotations, and source wording; do not invent missing content.
Preview the cleaned Markdown and report extraction limitations. If the user asked only to read or analyze it, keep the result transient.
Optional raw capture
When the user asks to retain the cleaned source:
- Resolve the user vault.
- Hash the exact cleaned bytes with SHA-256.
- Draft a new immutable payload such as
.raw/captured/<sha256>.mdand, when provenance metadata is needed, a new create-only sidecar with the normalized URL, retrieval date, extractor name and version, and content hash. - Use
expected_hashes: nullandmode: create. If that content-addressed payload already exists with the same bytes, report a no-op; never overwrite it. - Build, inspect, and apply one
claude-obsidian.transaction.v1capture bundle as described in operation-transactions.md.
Do not create wiki pages, update indexes, assess claims, or mark the source as
ingested. Invoke wiki-ingest as a distinct requested operation if the user
wants the captured payload incorporated into the knowledge base.
Frequently asked questions about Defuddle
Similar skills
Extracting with OCR
Efficiently extract text from image-based documents.
PaddleOCR Text Recognition
Extract text from images and PDFs with high accuracy.
PDFtk Server
Command-line tool for comprehensive PDF manipulation.
Latchshot Page Capture
Capture public web pages as images or PDFs easily.
Convert Word to Markdown
Effortlessly convert .docx files to Markdown format.
Convert PDF to Markdown
Transform PDF documents into Markdown for easier analysis.
