
DevOps Test Failures Analyzer
OfficialFreeUnderstand and track DevOps test failures effortlessly.
Free · Opens the source repo
What DevOps Test Failures Analyzer does
The DevOps Test Failures Analyzer skill is designed to help developers and DevOps teams analyze test failures and Code Analyzer violations with clarity and precision. This skill parses failure payloads and translates complex error messages into plain language, detailing the specific file, method, and line number where issues arise. It categorizes failures into types such as assertion failures, exceptions, and timeout errors, providing a clear understanding of what went wrong and why. By grouping failures by category, it allows users to quickly identify patterns and underlying issues.
In addition to identifying failures, the skill offers prioritized improvement suggestions based on the analysis of the failure messages. Each suggestion is actionable and specifies whether the fix should be made in the test code or the production code. This level of detail helps teams to not only address immediate issues but also to strengthen their testing practices over time. The suggestions are grounded in reasoning rather than code rewrites, ensuring that users know exactly what needs to change and why.
Furthermore, the skill includes an optional feature for creating tracked work items to log fixes. This is a confirmation-gated operation, meaning that users must explicitly request it, ensuring that work items are only created when genuinely needed. This functionality is particularly useful for teams that want to maintain a clear record of issues and resolutions, facilitating better project management and accountability.
Overall, the DevOps Test Failures Analyzer skill is a valuable tool for any developer or DevOps team looking to improve their testing processes and address failures efficiently. It streamlines the analysis of test results and provides actionable insights, making it easier to maintain high-quality code and robust testing practices.
When to use it
Use this skill when you encounter test failures or Code Analyzer violations and need to understand the root cause or improve your tests.
When not to use it
This skill is not suitable for generating new test code or writing fix code; use other tools for those tasks.
What you can build with it
Analyzing Test Failures
When a test run fails, use this skill to parse the failure payload and understand the root cause.
Improving Code Quality
After identifying failures, leverage the skill's suggestions to enhance your tests or production code.
Tracking Fixes
Create a work item to track the resolution of a failure, ensuring accountability within your team.
How to install DevOps Test Failures Analyzer
View source1. Install with the skills CLI
npx skills add forcedotcom/sf-skills/dx-devops-test-failures-analyze --agent claude-code2. Or install it manually
Download the skill folder and drop it into ~/.claude/skills/ for all projects, or .claude/skills/ to scope it to one repo. Restart Claude Code so it picks up the new skill.
Anthropic's agentic coding CLI, and the reference implementation of Agent Skills. Drop a skill folder into ~/.claude/skills and Claude Code loads it automatically whenever a task matches the skill's description. Claude Code docs
Inside SKILL.md
Written by forcedotcomAnalyze DevOps Center Test Failures
Parses a test failure or Code Analyzer violation payload, explains it in plain language, produces prioritized improvement suggestions, and — only on explicit user request — creates a tracked fix work item. Parts 1–2 are pure reasoning (no writes); Part 3 is an optional, confirmation-gated write.
Never expose raw JSON, stack traces, or internal Salesforce error codes to the user. Always translate to file name, method, line, and plain description.
Prerequisites
- Parts 1–2 (analysis): If the failure payload is already in context, no prerequisites are needed — this is pure reasoning. If you must fetch the payload yourself, run prerequisites (
references/prerequisite-checks.md, Prereqs 1–4) and obtain the execution result viadx-devops-test-suite-run(its polling step). - Part 3 (work item): Run Prerequisites 1–4. You also need a
DevopsProjectIdto file under and anOwnerId(assignee). Seereferences/work-item-creation.md.
Part 1 — Classify and explain each failure
Determine the failure category, then for each failure extract and translate to plain language: offending file/class, method, line number, the rule or assertion violated, and a fix direction (without writing code). Group failures by category if more than one.
| Category | Description |
|---|---|
| Assertion failure | A test assertion failed (expected vs actual mismatch) |
| Exception | An unhandled exception was thrown |
| Code Analyzer violation | A static-analysis rule was violated (e.g. ApexCRUDViolation) |
| Timeout | Test exceeded execution time limit |
| Compile error | Class failed to compile |
Output format:
Test failure summary:
<N> failure(s) found:
1. [<Category>] `<ClassName>.cls` — `<methodName>()` at line <N>
What happened: <plain-language description>
Rule violated: <ruleName or assertion description>
Fix direction: <plain-language suggestion>
Full category/pattern tables and Code Analyzer rule translations: references/failure-categories.md and references/code-analyzer-violations.md.
Empty / no-data case: If the payload contains no failures or violations, report that clearly (e.g. "No failures found in the provided execution results.") and stop. Do NOT fabricate failures or suggestions.
Part 2 — Improvement suggestions
Run this after execution completes with failures, not on static source. For each failed test, reason over the failure message (the primary signal) to identify what the test is not handling, then produce a specific, actionable suggestion and a fix location (Test vs Production code). The full failure-pattern → suggestion mapping is in references/failure-categories.md.
Test improvement suggestions based on execution results:
`<testMethodName>()` — [Assertion Failure / Exception / etc.]
Failure: "<failure message>"
What this reveals: <plain-language explanation>
Suggestion: <specific, actionable recommendation>
Fix location: Test | Production code
Overall: <N> improvement(s) across <M> failed test(s).
Do not rewrite the test — only describe what needs to change and why. Fix location: Production code indicates a code defect exposed by a sound test (track separately, not a test-quality blocker). Fix location: Test indicates the test needs hardening (setup, assertions, edge cases).
Part 3 — Create a fix work item (optional, on request only)
Trigger only when the user wants to create a fix work item, log a remediation, or assign a failure to a developer. This is a write operation with a mandatory confirmation gate. Follow references/work-item-creation.md for inputs, the subject/assignee/project confirmation gate, the sf data create record --sobject WorkItem call, and error handling.
Use
WorkItem(no namespace) —DevopsWorkItemis not a supported sObject in this org version.
If no DevopsProject exists in the org, report that the work item cannot be created until a project is set up — do NOT fabricate a project or proceed.
Related skills
dx-devops-test-suite-run— produces the failure payload (via its polling step) that feeds this skill.dx-devops-test-suite-assignments-configure— assign/strengthen the suites whose tests are failing.platform-apex-generate/platform-apex-test-generate— to actually write fix code or new test classes (out of scope here).
Frequently asked questions about DevOps Test Failures Analyzer
Similar skills
Turborepo
Optimized build system for JavaScript/TypeScript monorepos.
Azure Pipelines Validation
Streamline your Azure DevOps pipeline changes locally.
Azure Developer CLI
Streamline your Azure project workflows with best practices.
Azure Container Registry CLI
Manage Azure Container Registry resources with ease.
Aspire
Build and orchestrate polyglot distributed applications seamlessly.
Vercel CLI
Manage and deploy Vercel projects from the command line.
