Diagnose Claude skill hints as a sequence of observable steps
An installed skill can remain unused even when its instructions look relevant. Before rewriting the skill, establish where the request stops: installation, hook registration, prompt matching, context delivery, skill invocation or the work itself. Those are different failures and need different fixes.
The November 11 release of the community claude-skills repository provides a useful example. Version 2.14.11 adds prompt-time hints across eight plugins to draw attention to selected existing skills. Its maintainer reports improved activation on a limited evaluation set. That is a reason to examine the mechanism, not evidence that it will improve every workflow. See the release announcement.
This guide builds an original troubleshooting process around a source review. We have not run a live benchmark of these hooks. The examples below are proposed checks for your own controlled environment, not reported test results. They focus on the engineering plugin configuration at that version, rather than assuming every plugin or future release behaves identically.
If you are new to the distinction between discovering instructions and executing a task, start with how agents discover and activate skills. Here, the practical question is narrower: what evidence would justify changing the router, the skill or neither?
Separate the hint from the skill invocation
Claude Code's hook reference documents UserPromptSubmit as an event before prompt processing. A hook can provide additional context through its structured output. Adding context is not equivalent to invoking a skill or proving that the resulting work followed its instructions.
Use three distinct observations in your notes. First, record whether the hook ran. Second, record whether a relevant hint reached the host. Third, record whether the intended skill was invoked. A successful process with no hint is different from a delivered hint that the model did not act on.
Then assess the work. A skill can load and still produce an incomplete patch, an unsuitable plan or an answer outside the requested scope. Counting activation alone would conceal that failure. Equally, a competent answer produced without a skill does not prove that your discovery configuration works. Decide whether your experiment measures routing, task quality or both before you collect results.
This distinction also helps when a teammate reports that the skill is broken. Ask for the missing observation rather than another screenshot of the final answer. A short record of the sequence is usually more useful than an entire conversation pasted into an issue.
Inspect the installed plugin before changing it
Read the files from the version you actually installed. A repository's current default branch can differ from your local plugin. Record the version or revision alongside your test, and compare the relevant registration, router and route configuration as a set.
The reviewed engineering hook registration starts the bundled Python router on UserPromptSubmit with a five-second timeout. That creates a concrete diagnostic boundary: registration, interpreter availability and script location must work before prompt matching matters.
Check those prerequisites without changing unrelated settings. If the host never starts the hook, changing a route expression cannot fix it. If a different plugin copy is active, editing a checkout elsewhere may have no effect. If your environment cannot run the interpreter, a polished skill description will not make the script execute.
Keep the first investigation read-only. Review the installed code and host configuration before running bundled scripts. The agent skills security guide explains why a skill's instructions and supporting code deserve separate attention. Repository popularity does not replace that review.
Build a small synthetic prompt set
Choose one task family and write a few prompts that represent it. Avoid real customer material, credentials or internal incident details. Synthetic examples make it easier to share findings and repeat the experiment without carrying private conversation history into the result.
For a test-writing skill, a useful first set contains a straightforward request, a paraphrase, an adjacent debugging request and an unrelated request. Decide which should receive a hint before looking at the router's output. Otherwise, it is easy to redefine success around whatever the implementation already does.
Here is an illustrative record format. It is a local test-plan example, not a configuration file consumed by Claude Code:
case: new-unit-tests
prompt: "Write unit tests for a small price formatter."
expected_hint: relevant test-writing skill
expected_invocation: inspect after hint delivery
fixture: synthetic formatter with documented behaviour
session: fresh
result: not yet run
Add a second case that asks why an existing test fails. Both cases involve tests, but they may belong to different workflows. A useful router should not automatically treat every mention of a shared noun as the same intent. Your expected result should reflect the skill's actual scope.
Keep this set small enough to inspect manually. The goal of the first pass is to find the failing boundary. A large spreadsheet of unexplained misses creates more work without making the underlying problem clearer.
Read positive matches and exclusions together
The pinned engineering route configuration combines positive expressions with exclusions. For its test-writing route, certain debugging and end-to-end terms can suppress an otherwise plausible match. This is a deliberate distinction in the configuration, not proof that a skill is missing.
When a prompt surprises you, inspect the complete matching rule. A phrase may satisfy a positive expression while another word excludes the route. Editing only the positive expression can leave the miss unchanged. Removing every exclusion can create unwanted hints for neighbouring tasks.
Treat a proposed rule change as a trade-off. Write down the desired new match and at least one neighbouring request that should remain unaffected. Review both before accepting the change. If the desired distinction cannot be expressed clearly, the task scope may need clarification before the pattern does.
Avoid rewriting a user's natural request solely to win a routing test. That can be a useful diagnostic probe, but it does not improve the original experience. Preserve the original synthetic case and add the rewritten version as a separate case. The difference between them is evidence about the route, not evidence that the first request was wrong.
Account for session state and prompt shape
The reviewed router implementation skips slash-prefixed prompts and uses only the beginning and end of long prompts for matching. It also records suggested skills in temporary session state to avoid repeating them. Errors can produce silence with a successful exit status.
These details matter when designing a comparison. A second prompt in an existing session may not be equivalent to the first prompt in a fresh session. A long pasted document may place the relevant request outside the text the router examines. An explicit command may follow a different path from an ordinary conversational request.
Run a fresh-session case and a repeated-session case separately. Label them clearly. Do not delete unrelated temporary files or reset a user's working environment merely to make a test cleaner. Use an isolated test session and follow the host's normal session controls.
For a long-prompt case, keep the task request stable and vary where the supporting material appears. Record the observed change without generalising it to the model's ability to understand long context. You are probing the router's matching input, which is a smaller and more specific question.
Compare routing outcomes without inventing a benchmark
Use a simple matrix that keeps the stages separate. The following rows are hypothetical classifications, not measurements from the repository or from our testing:
| Hook ran | Hint delivered | Skill invoked | Interpretation |
|---|---|---|---|
| No | No | No | Investigate registration and execution prerequisites |
| Yes | No | No | Inspect matching, state and silent error paths |
| Yes | Yes | No | Inspect host output handling and subsequent model behaviour |
| Yes | Yes | Yes | Evaluate whether the task outcome meets the brief |
Record an unknown when you cannot observe a stage. Do not turn missing evidence into a negative result. For example, an interface that hides hook output does not establish that no output was produced.
Hold the model, plugin revision and fixture steady while comparing a routing change. If you also upgrade the model, rewrite the skill and change the prompt, a better answer cannot tell you which change helped. Make one bounded change, repeat the relevant cases and preserve the prior observations.
For a broader task-quality comparison after routing works, use the evaluation approach in our Fable 5.2 skill guide. Keep that experiment separate from a claim that a hint was emitted.
Troubleshoot the failure you actually observed
The hook never appears to run
Start with registration and the active plugin version. Confirm that the host can find the configured command and its interpreter. Check the relevant host diagnostics using a synthetic prompt. Do not assume that a file's presence in a repository means it is registered in the running client.
The process succeeds but nothing is suggested
Inspect route eligibility, session history and error handling. Success can mean that the router chose to remain silent. In the reviewed implementation, it can also mean an error was handled without failing the prompt. A zero exit status therefore answers only a narrow process question.
The wrong skill is suggested
Compare the competing scopes and matching rules. Keep a minimal synthetic reproduction with enough context to preserve the ambiguity. A broad rule that improves one case may steal requests from another specialist workflow. Include the neighbouring case in your review before widening the pattern.
The skill loads but the work is poor
Move the investigation to the skill instructions and task fixture. Check whether required inputs were available and whether the output meets the request. Repeatedly changing routing rules cannot repair an instruction that omits a necessary validation step. The SKILL.md format guide is a better reference for that layer.
Decide whether a hint is worth keeping
A hint is useful when it helps a relevant skill receive attention without creating distracting suggestions elsewhere. Evaluate both sides. Keep examples where the hint helped, but also inspect near misses and requests that should be handled directly. An increased invocation count is not automatically an improvement.
Write the acceptance decision in practical terms. For example, your team might require a relevant suggestion on its representative test-writing tasks and no additional suggestion on its selected debugging cases. That is a local acceptance rule, not a universal threshold for every repository.
Before rolling out a change, preserve the previous configuration and document how your team will return to it. Follow the plugin's documented controls for disabling the router when needed. Avoid turning off unrelated hooks to solve a problem confined to one routing mechanism.
Finally, keep the resulting note short enough to use later: installed revision, synthetic case, observed stage, specific adjustment and outcome. That record makes the next upgrade easier to assess. It also lets you distinguish a new regression from an old limitation without reconstructing an entire conversation.
