What is Cost Diff used for?

Cost Diff is used to compare two JSON snapshots of cost summaries to detect any changes in spending associated with code changes.

How does Cost Diff work?

It loads two JSON files, performs sanity checks, and outputs a report detailing the differences in costs, categorized by tier and model.

Can I set thresholds for cost increases?

Yes, you can set thresholds for both percentage and absolute cost increases, which can automatically fail PRs if exceeded.

What types of changes does Cost Diff report?

It reports changes as added, removed, or changed, allowing teams to see the most significant impacts on costs.

Is Cost Diff suitable for all development teams?

It is particularly useful for teams that need to monitor and control costs, but may not be necessary for teams without cost tracking requirements.

Cost Diff

Name: Cost Diff
Availability: InStock
Author: ruvnet

Free

Detect cost regressions between JSON snapshots.

by ruvnet67.6k stars on ruvnet/ruflo

Updated Aug 10, 2026

Get this skill

Free · Opens the source repo

What Cost Diff does

Cost Diff is a specialized tool designed for developers and teams who need to monitor and control costs associated with their software projects. By comparing two specific JSON snapshots generated by cost-summary outputs, it allows users to pinpoint changes in spending at a granular level. This capability is particularly useful in a continuous integration context, where understanding the financial impact of code changes is critical. The skill helps answer the question, "Did this pull request (PR) add to our spending compared to the main branch?".

The tool operates by loading two JSON files: a baseline snapshot and a current snapshot. It performs a series of checks to ensure both files contain the necessary data, such as total costs and session counts. The output is a detailed report highlighting the differences in spending by tier and model, categorizing each change as added, removed, or changed. This categorization helps teams quickly identify which areas of their code are impacting costs the most.

Cost Diff is particularly beneficial for teams practicing cost-conscious development, as it integrates seamlessly into PR workflows. By setting thresholds for acceptable cost increases, teams can automatically fail PRs that exceed these limits, ensuring that only cost-effective changes are merged into the codebase. The tool also includes features to alert on specific classes of costs, allowing for a more nuanced understanding of spending patterns.

In summary, Cost Diff is an essential tool for any development team looking to maintain control over their project budgets while ensuring that new code contributions do not inadvertently increase costs. Its detailed reporting and integration into CI workflows make it a valuable addition to any cost management strategy.

When to use it

Use Cost Diff when you want to evaluate the cost impact of specific code changes before merging them into your main branch.

When not to use it

This tool is not suitable for teams that do not track costs or do not require detailed financial oversight of their development activities.

What you can build with it

Integrating into CI/CD

Incorporate Cost Diff into your CI/CD pipeline to automatically check for cost regressions before merging PRs.

Monitoring Spending Trends

Use Cost Diff to analyze spending trends over time by comparing snapshots from different development stages.

Cost Management Strategy

Implement Cost Diff as part of a broader cost management strategy to ensure that new features do not exceed budget constraints.

How to install Cost Diff

View source

1. Install with the skills CLI

npx skills add ruvnet/ruflo/cost-diff --agent claude-code

2. Or install it manually

Download the skill folder and drop it into ~/.claude/skills/ for all projects, or .claude/skills/ to scope it to one repo. Restart Claude Code so it picks up the new skill.

Anthropic's agentic coding CLI, and the reference implementation of Agent Skills. Drop a skill folder into ~/.claude/skills and Claude Code loads it automatically whenever a task matches the skill's description. Claude Code docs

Inside SKILL.md

Written by ruvnet

PR-level cost regression detection. Where cost-counterfactual compares to HYPOTHETICAL baselines (always-haiku/sonnet/opus) and cost-burn compares latest bucket to PRIOR MEAN, cost-diff compares two SPECIFIC known-good snapshots.

Question	Skill
"What would we have spent at always-X?"	`cost-counterfactual`
"Is daily burn accelerating vs prior mean?"	`cost-burn`
"Did THIS PR add spend vs main?"	`cost-diff` ← this

Algorithm

Implementation: scripts/diff.mjs. Consumes the stable JSON contract from cost summary --format json.

Load --baseline and --current JSON snapshots.
Sanity check: both must have total_cost_usd + sessionCount (cost-summary shape).
Per-key delta: byTier (haiku/sonnet/opus) and byModel (each model).
Each entry tagged added / removed / changed based on baseline / current zero-ness.
Sort table by |delta| descending so the biggest movers are at the top.
--alert-on-pct N: exit 1 when total_pct > N.
--alert-on-usd N: exit 1 when total_delta_usd > N. Both can be set; first to trigger wins.

PR-gate workflow

# Capture baseline (e.g. on main, via the cost-tracker-smoke CI workflow)
cost summary --format json > baseline.json

# On the PR branch, capture current state
cost summary --format json > current.json

# Compare; fail the PR if total spend grew >10% OR >$5
cost diff --baseline baseline.json --current current.json \
          --alert-on-pct 10 --alert-on-usd 5.00

The combination of both flags catches:

Percent-only fires: a small absolute change but a meaningful shift (e.g. doubling from $0.10 to $0.20 hits +100% but only +$0.10).
USD-only fires: a large absolute change with a small percent (e.g. growing from $100 to $110 is only +10% but +$10).

Either signal can fail the PR independently — they're OR'd.

--alert-on-class-pct (iter 86)

The two USD-level thresholds above miss a regression class: when ONE token type grows disproportionately even though total spend grows modestly. Example: a PR introduces a verbose context-cache pattern, total spend grows only 10% (under --alert-on-pct 50), but cache_write tokens grow 900%. The iter-82 driver hides inside the USD signal.

--alert-on-class-pct cache_write:50 exits 1 when cache_write tokens grow more than 50% baseline → current. Multiple classes can be checked in one flag (comma-separated):

cost diff --baseline baseline.json --current current.json \
          --alert-on-class-pct cache_write:50,output:25

First class to breach wins. Valid classes: input | output | cache_write | cache_read.

Recommended PR-gate triad:

cost diff --baseline ... --current ... \
          --alert-on-pct 25 \
          --alert-on-usd 5.00 \
          --alert-on-class-pct cache_write:100

Three orthogonal signals — pct (total grew), usd (large absolute jump), class-pct (composition shifted). Each catches what the others miss; AND-of-OR semantics means any one firing fails the PR.

Smoke transcript (synthetic baseline + current)

| Total spend       | $1.000000 | $1.500000 | +$0.500000 | 50.00% |
| Sessions          | 10        | 13        | +3         | 30.00% |

## By tier
| opus   | $0      | $0.60   | +$0.600000 | new      | added   |
| sonnet | $0.70   | $0.50   | -$0.200000 | -28.57%  | changed |
| haiku  | $0.30   | $0.40   | +$0.100000 | 33.33%   | changed |

Notice the table is sorted by absolute delta, not alphabetically — the biggest mover (opus newly added) bubbles to the top. Operators reading top-down see "what mattered" first.

Exit codes

Exit	Meaning
0	No alert, OR no thresholds set
1	--alert-on-pct or --alert-on-usd threshold exceeded
2	Config error (missing files, invalid JSON, malformed snapshot)

Status column

Status	Meaning
`added`	This tier/model was $0 in baseline, >$0 in current
`removed`	This tier/model was >$0 in baseline, $0 in current
`changed`	Both baseline and current >$0; delta is the difference

Entries with baseline === 0 && current === 0 are dropped (nothing to report).

Composition with cost-summary

cost-diff is the SECOND HALF of a contract that cost-summary started: the stable JSON shape from cost summary --format json. Both pieces have been frozen — adding fields to summary is fine; renaming or removing isn't.

If you're consuming snapshots elsewhere (dashboards, alerting), the same shape works — cost-diff is just one consumer.

Frequently asked questions about Cost Diff

Similar skills

Turborepo

Optimized build system for JavaScript/TypeScript monorepos.

Azure Pipelines Validation

Streamline your Azure DevOps pipeline changes locally.

Azure Developer CLI

Streamline your Azure project workflows with best practices.

Azure Container Registry CLI

Manage Azure Container Registry resources with ease.

Aspire

Build and orchestrate polyglot distributed applications seamlessly.

Vercel CLI

Manage and deploy Vercel projects from the command line.