
Spec-to-Code Compliance
FreeEnsure code aligns with its documentation requirements.
Free · Opens the source repo
What Spec-to-Code Compliance does
Spec-to-Code Compliance is a specialized skill designed to identify discrepancies between software documentation and the actual implementation in code. This tool is particularly useful for developers and teams working with authoritative documentation, such as whitepapers, protocol specifications, or design documents. By analyzing both the code and the corresponding documentation, it highlights gaps where the implementation does not meet the specified requirements, helping to pinpoint potential bugs or necessary documentation updates.
The skill operates by breaking down the documentation into individually checkable requirements and dispatching agents to analyze the code against these requirements. Each requirement is assessed for compliance, and the results are compiled into a detailed report. This report categorizes findings into various verdicts, such as implemented, partial, contradicted, stronger-than-spec, absent, or undecidable. This structured approach allows users to understand not just whether the code meets the documentation but also to discern the implications of any discrepancies found.
This skill is particularly valuable in environments where documentation is critical for compliance or auditing purposes. It provides a systematic way to ensure that the software behaves as intended and that any deviations are documented and addressed. By using this tool, teams can improve their code quality and maintain alignment with their intended design and operational specifications.
However, it's important to note that this skill is not intended for use with code that lacks documentation. Without a clear set of expectations to compare against, the analysis cannot yield meaningful results. Additionally, while it can identify discrepancies, it is not a general-purpose bug-finding tool; it focuses specifically on the relationship between the code and its documentation, rather than uncovering all potential bugs in the codebase.
When to use it
Use this skill when you have authoritative documentation and code that should adhere to it, such as when validating a design document against the actual implementation.
When not to use it
Avoid using this skill for undocumented code or for general bug-finding, as it specifically targets discrepancies between code and its documentation.
What you can build with it
Validating a Protocol Implementation
When implementing a new protocol, use this skill to ensure that your code adheres to the specifications outlined in the protocol documentation.
Auditing Software for Compliance
In a regulated environment, use this tool to verify that your software meets the necessary compliance requirements as stated in official documents.
Updating Legacy Code
When updating or refactoring legacy code, use this skill to check that the changes still align with the existing documentation.
How to install Spec-to-Code Compliance
View source1. Install with the skills CLI
npx skills add trailofbits/skills/spec-to-code-compliance --agent claude-code2. Or install it manually
Download the skill folder and drop it into ~/.claude/skills/ for all projects, or .claude/skills/ to scope it to one repo. Restart Claude Code so it picks up the new skill.
Anthropic's agentic coding CLI, and the reference implementation of Agent Skills. Drop a skill folder into ~/.claude/skills and Claude Code loads it automatically whenever a task matches the skill's description. Claude Code docs
Inside SKILL.md
Written by trailofbitsSpec-to-Code Compliance
Two artifacts disagree, and the job is to find where. The documentation says what the system does; the code decides what it actually does. Every gap between them is either a bug or a documentation fix, and which one it is is the finding.
When to Use
You have both documentation describing intended behavior and the code that should implement it. A whitepaper against a protocol, a design note against a service, a README's stated guarantees against the functions behind them.
Most useful when the document is authoritative — something a client wrote, published, or is audited against — because then a divergence is a defect rather than stale prose.
When NOT to Use
Not for code with no documentation of intended behavior. There is nothing to check against, and a requirement
inferred from the code is checked against itself. Build the system model first with audit-context-building.
Not for finding bugs in general. This finds one class: where the code and the document disagree. A bug both artifacts are silent about is out of scope, and a bug the document endorses is a finding against the document.
Not for writing or improving documentation, though it produces the list of what needs fixing.
Do not check requirements in this context
Run /spec-to-code-compliance:spec-compliance <path>. The slash command takes a path; to name the
specification directly or widen the fan-out, ask for the run with those values — "run spec-compliance on
./contracts against SPEC.md, checking 20 requirements" — and they reach the script as {path, spec, limit}.
Typing the object literally after the slash command does not work; it arrives as a string and becomes the path.
It finds the documents, splits them into individually checkable requirements, gives each requirement its own
agent to hunt the code with, has independent agents try to refute every divergence before it is reported, and
writes spec-compliance/REPORT.md plus one file per requirement under spec-compliance/requirements/. Only
compact records come back here.
For a single requirement, dispatch the spec-to-code-compliance:spec-compliance-checker agent at it.
This is not a preference about where output lands. The check does not fit in one context window if it is done honestly: judging one requirement means reading the enforcement, its callees, and its callers, and doing that for thirty requirements means holding thirty call chains at once. Attempted inline, the first few get a real check and the rest get a plausible one — and the transcript looks the same either way, because a verdict resting on a promising function name reads exactly like one resting on having read the function. Per requirement, in its own context, is what makes that difference visible.
Two properties come from the script rather than from instructions, and cannot be had here:
- A refutation the finding's author did not perform. Claude favors findings it produced when asked to check them. The workflow sends each divergence to agents that did not produce it — one reading the code again, one re-reading the document — and drops what either knocks down.
- Records that cannot be prose. A subagent bound to a return schema has to name the lines it read and the
searches it ran. An
absentverdict arrives with the patterns tried and their results attached, which is the only thing separating a real absence from a search that stopped early.
Measured on the routes-not-inline eval: with this plugin installed the work is dispatched every run, without
it never — Δ +1.00. Deleting this section while leaving the workflow in place changes nothing, because the
workflow is a real command that gets found and dispatched on its own. Read that as the mechanism carrying the
behavior rather than this text: the section is here so a human knows what runs and why, not because the routing
depends on it.
What comes back, and how to read it
Every requirement gets one of six verdicts: implemented, partial, contradicted, stronger-than-spec,
absent, or undecidable. The interesting ones are the middle four.
partialis usually the most serious thing in the report. The requirement holds on the paths anyone would test and fails on one nobody did, which is how it survived long enough to be found.absentrests entirely on itssearchedrecord. Read it. Enforcement often lives somewhere the search did not go — a modifier, a base class, a caller that checks first.undecidable, and anydocumentProblem, are findings about the documentation. A requirement too vague to check is one the client cannot hold anyone to.stronger-than-specis an undocumented constraint. It works today, and nothing tells the next person changing that code that anything depended on it.
The report also carries the reverse direction — behavior the code has that no document mentions — which the per-requirement pass cannot find by construction, since it is driven by the documents.
Read notChecked, unverified, and unreadableDocuments before treating the report as complete. Requirements
below the fan-out cut were never checked, and a divergence whose refutation agents both failed is unverified
rather than confirmed.
Judgment the workflow does not make for you
Severity is consequence, not distance from the text: DIVERGENCE_RUBRIC.md. A rounding step that bleeds a pool outranks a MUST satisfied by different means than the document describes, and documentation drift with no behavioral consequence is a docs ticket.
The verdict is not the finding. absent on a mandatory requirement is a finding; absent on a sentence
describing a roadmap item is not. Deciding which is which is what this skill is for, and the workflow hands you
the evidence to decide it with.
The target does not have to be a contract
The question is the same everywhere — what does this requirement demand, where would it be enforced, is it enforced on every path — but what counts as a specification and where enforcement hides both change. DOMAIN_NOTES.md maps that across contracts, C and C++, services, and decompiled firmware, and covers the case where the specification is an RFC or a standard rather than a project document. Read it when the target is not a contract, and when scoping a check against a large standard.
Reference
- ANALYSIS_FORMAT.md — the on-disk format for a per-requirement analysis. Read when extending this plugin or judging whether a record is trustworthy.
- WORKED_EXAMPLE.md — three requirements chased to a verdict, one per verdict that is easy to get wrong: arithmetic that satisfies a requirement it does not resemble, an absence whose searches are the finding, and enforcement present on every path but the one nobody tested.
- DOMAIN_NOTES.md — per-domain mapping of specifications and enforcement.
- DIVERGENCE_RUBRIC.md — severity, and the two directions of a gap.
Frequently asked questions about Spec-to-Code Compliance
Similar skills
Quality Playbook Generator
Run comprehensive quality audits on any codebase.
PR Draft Summary
Automate PR summary generation for openai-agents-python.
Final Release Review
Streamline your release candidate audits with ease.
Unit Test Vue Pinia
Efficiently write and review unit tests for Vue 3 applications.
Slang Shader Expert
Optimize and integrate Slang shaders with ease.
Telemetry Standards
Ensure consistent event tracking in Supabase Studio.
