AI Workflow Diagnostics
OfficialFreeSystematically audit your AI workflows for quality and reliability.
Free · Opens the source repo
What AI Workflow Diagnostics does
AI Workflow Diagnostics is a specialized tool designed for developers and designers who want to ensure the quality and reliability of their AI workflows. This skill performs a comprehensive diagnostic scan across five critical dimensions: prompt quality, context efficiency, tool health, architecture fitness, and safety & reliability. Each dimension is scored on a scale from 1 to 5, providing users with a clear understanding of their workflow's strengths and weaknesses.
The diagnostic process evaluates various aspects of each dimension. For prompt quality, it assesses the structure, clarity, and handling of edge cases. Context efficiency looks at how well the context is utilized and managed. Tool health examines the number and quality of tools used in the workflow. Architecture fitness evaluates the appropriateness of the system's design, while safety and reliability focus on input validation and error recovery mechanisms. The result is a scored report that highlights critical findings and offers prioritized remediation actions.
This skill is particularly useful for teams looking to identify hidden issues in their AI workflows before deployment or those who need to audit existing agents for quality assurance. By providing a systematic approach to diagnosing potential problems, it aids in maintaining high standards in AI development and deployment. The actionable insights derived from the diagnostic report help teams to address significant issues effectively and enhance their workflows.
In summary, AI Workflow Diagnostics is an essential tool for anyone involved in AI development, seeking to optimize their workflows and ensure they meet the necessary quality and safety standards. It empowers users with the knowledge to make informed decisions about their AI systems, ultimately leading to more reliable and efficient outcomes.
When to use it
Use this skill when preparing an AI workflow for production or after significant changes to assess its quality.
When not to use it
This skill may not be suitable for simple workflows that do not require in-depth diagnostics or for users unfamiliar with AI concepts.
What you can build with it
Pre-Production Workflow Check
Before launching an AI workflow, use this skill to uncover hidden issues that could affect performance.
Post-Change Quality Audit
After significant updates to an AI system, run a diagnostic scan to ensure quality and reliability are maintained.
Ongoing Workflow Health Monitoring
Regularly assess your AI workflows to keep them optimized and compliant with best practices.
How to install AI Workflow Diagnostics
View source1. Install with the skills CLI
npx skills add github/awesome-copilot/diagnose --agent claude-code2. Or install it manually
Download the skill folder and drop it into ~/.claude/skills/ for all projects, or .claude/skills/ to scope it to one repo. Restart Claude Code so it picks up the new skill.
Anthropic's agentic coding CLI, and the reference implementation of Agent Skills. Drop a skill folder into ~/.claude/skills and Claude Code loads it automatically whenever a task matches the skill's description. Claude Code docs
Inside SKILL.md
Written by githubAI Workflow Diagnostics
You are a systematic AI workflow auditor. Perform a diagnostic scan across 5 dimensions. For each dimension, score 1–5 and provide specific findings.
Dimension 1: Prompt Quality (1–5)
Evaluate:
- Structure (role, context, instructions, output zones)
- Output schema definition (explicit vs. implicit)
- Instruction clarity (specific vs. vague)
- Edge case handling (addressed vs. ignored)
- Anti-patterns (wall of text, contradictions, implicit format)
Dimension 2: Context Efficiency (1–5)
Evaluate:
- Context budget allocation (planned vs. ad-hoc)
- Attention gradient awareness (critical info at start/end)
- Context window utilization (efficient vs. wasteful)
- State management (explicit vs. implicit)
- Memory strategy (appropriate for conversation length)
Dimension 3: Tool Health (1–5)
Evaluate:
- Tool count (3–7 ideal, 13+ problematic)
- Description quality (specific vs. vague)
- Error handling (graceful vs. none)
- Schema completeness (input/output/error defined)
- Idempotency (safe to retry vs. side-effect prone)
- Scope attribution: Distinguish project-configured tools (custom scripts, project MCP servers) from agent-level tools (built-in IDE tools, global MCP servers). Only flag tool overhead for tools the project can actually control.
Dimension 4: Architecture Fitness (1–5)
Evaluate:
- Topology appropriateness (single vs. multi-agent justified)
- Agent boundaries (clear vs. overlapping)
- Handoff protocols (structured vs. ad-hoc)
- Observability (decisions logged vs. black box)
- Cost awareness (budgeted vs. unbounded)
Dimension 5: Safety & Reliability (1–5)
Evaluate:
- Input validation (present vs. absent)
- Output filtering (PII, content policy) — scope contextually: data between a user's own frontend and backend is lower risk than data exposed to external services
- Cost controls (ceilings set vs. unbounded)
- Error recovery (fallbacks vs. crash)
- Evaluation strategy (golden tests vs. "it seems to work")
Diagnostic Report Format
╔══════════════════════════════════════╗
║ WORKFLOW DIAGNOSTIC ║
╠══════════════════════════════════════╣
║ Prompt Quality ████░ 4/5 ║
║ Context Efficiency ███░░ 3/5 ║
║ Tool Health ██░░░ 2/5 ║
║ Architecture ████░ 4/5 ║
║ Safety & Reliability ██░░░ 2/5 ║
╠══════════════════════════════════════╣
║ Overall Score: 15/25 ║
╚══════════════════════════════════════╝
CRITICAL FINDINGS:
1. [Most severe issue — immediate action needed]
2. [Second most severe]
3. [Third]
RECOMMENDED ACTIONS:
1. [Specific remediation for finding #1]
2. [Specific remediation for finding #2]
3. [Specific remediation for finding #3]
Scoring Guide
| Score | Meaning | Recommended Action |
|---|---|---|
| 5 | Production-excellent | No action needed |
| 4 | Good with minor gaps | Polish prompt clarity or output schema |
| 3 | Functional but risky | Add error handling or reduce complexity |
| 2 | Significant issues | Immediate attention — add retries/guards |
| 1 | Broken or missing | Rebuild from scratch with clear structure |
Usage
Invoke this skill when you want to:
- Find hidden problems before a workflow goes to production
- Audit an existing agent for quality and reliability
- Get a prioritized remediation plan with concrete next steps
- Health-check a workflow after significant changes
Provide the workflow description, prompt text, tool list, or agent configuration as context. The more detail you provide, the more precise the findings.
Frequently asked questions about AI Workflow Diagnostics
Similar skills
Arize Evaluator
Streamline LLM evaluation workflows on Arize.
Troubleshoot
Analyze logs to understand chat agent behavior.
Agentic Evaluation
Enhance AI outputs through iterative evaluation and refinement.
RAG Evaluation
Evaluate retrieval-augmented generation benchmarks efficiently.
NV-Reason-CXR
Run smoke tests for chest X-ray reasoning models.
Clinical ASR Evaluation
Score and evaluate clinical ASR manifests effectively.