New to Claude Skills? Learn how to install them →

Claude Fable 5.2: how to evaluate your existing agent skills

A practical evaluation plan for existing agent skills after the Fable 5.2 and Mythos 5.2 announcement, with clear checks for behaviour and review effort.

November 10, 2026
Get Claude Skills
9 min read

What the Fable 5.2 announcement means for skill users

Anthropic announced Claude Fable 5.2 and Mythos 5.2 on 9 November 2026. It describes them as the same underlying model with different safeguards and access arrangements: Mythos is available through verification programmes for eligible cybersecurity and biomedical work. The announcement positions these models for demanding technical tasks and continues to recommend Opus 5.5 as a starting point for most everyday coding and knowledge work. Anthropic also reports improved speed and capabilities, but those are vendor results rather than measurements of your workflow. Read the primary announcement.

For an engineer maintaining a collection of skills, the immediate question is practical: which existing tasks justify trying a different model, and what evidence would support a switch? Installing more instructions or changing every default at once makes that question harder to answer. Start with one workflow whose result you can inspect.

This guide proposes an evaluation process, not a benchmark or a claim that we tested these models. It uses synthetic examples and focuses on the interaction between instructions, tools and final outputs. The agent skills overview provides background if your team needs a shared definition before comparing results.

Choose a task whose outcome you can judge

Pick an existing skill that solves a recurring problem. Good candidates have a clear output and someone who knows what a correct result looks like. For example, a documentation skill might produce a patch explaining how to configure a small service. A review skill might identify a particular class of missing validation in a code change.

Avoid beginning with an enormous open-ended project. If the result takes several days to inspect, you will struggle to tell whether a difference came from the model, the instructions or a change in scope. A bounded task gives you a useful comparison sooner and makes failed attempts easier to diagnose.

Write the acceptance criteria before the first run. For the documentation example, they might require an accurate environment-variable table, a working local setup sequence and a clear explanation of one failure mode. Specify what the skill should do when information is missing. Asking a useful question can be the correct outcome; inventing an answer should not receive credit merely because it completes the document.

Keep the first evaluation connected to a real need. A synthetic task should resemble the work you care about, while removing private information and unnecessary complexity. An artificial puzzle can be interesting without telling you much about whether your team's daily workflow should change.

Freeze the instructions and inputs

Record the exact skill revision used in the comparison. Keep its supporting files, scripts and references consistent. If the instructions change between runs, you are evaluating a combined change, not only a new model. That can still be useful, but label it honestly.

Create a clean copy of the same example project for each run. Reusing a directory after the first agent has modified it can give the second run an unintended advantage. It can also conceal destructive behaviour because the original input no longer exists. Preserve the starting state in a location appropriate for your existing development process.

The SKILL.md format guide can help you identify which parts of the workflow are instructions and which are supporting material. Include referenced files in your review. A stable top-level document does not establish a stable workflow if a linked script or reference changes underneath it.

Use an evaluation note such as the following. This is an illustrative record, not a supported model configuration file or an installation command.

Evaluation record
Task: document the setup of a synthetic example service
Skill revision: record the exact revision used
Input revision: record the clean starting state
Model and settings: record what the client actually used
Tools: list the tools available to the run
Expected result: accurate setup patch and named failure mode
Missing information: request clarification instead of guessing
Final verification: inspect the patch and check its instructions
Decision: retain, revise, expand the pilot, or stop

Check access before interpreting a failed run

Confirm that your chosen client actually offers the intended model to your account. A launch announcement and an integration's availability are separate pieces of evidence. Do not infer support in every editor or hosted service from the existence of the model. Record what the client shows and consult its current documentation for configuration details.

If a run fails before the task begins, classify that failure accurately. Missing access, an unsupported setting or a connection problem is not evidence that the model cannot perform the task. Resolve the setup issue and repeat the same input if it remains worth evaluating. Keep the failed attempt in the notes so the operational cost is not hidden.

Avoid copying an unfamiliar model identifier into configuration simply because it resembles an earlier version. This guide intentionally does not provide an API identifier or switching command. Use the official instructions for the specific client and account you are evaluating, then record the selected model as part of the result.

Separate discovery, execution and correctness

A useful evaluation follows three checkpoints. First, did the agent use the intended skill? Second, did it complete the permitted workflow? Third, did the final output satisfy the acceptance criteria? A failure at each checkpoint suggests a different next step.

If the skill was not selected, inspect its description and the user's request. If it loaded but a tool failed, investigate the tool or environment before rewriting the task instructions. If the workflow completed but produced an incorrect result, inspect the reasoning embodied in the output and the checks that should have caught the error.

The skill discovery guide explains the distinction between discovering instructions and carrying out their contents. For this comparison, rely on whatever observable evidence your client provides. Do not claim a skill activated merely because the final answer sounds similar to the skill's style.

Record uncertainty where the client does not expose a checkpoint. “Activation not observable” is more accurate than a guessed success. You can still inspect the final work, but the missing observation limits what you can conclude about the instructions themselves.

Use ordinary, incomplete and out-of-scope requests

Build a small task set with different failure opportunities. Include an ordinary request, one missing a necessary input and one outside the intended scope. The point is to test the boundaries you rely on, not to find a phrasing that produces an impressive demonstration.

For a setup-documentation skill, an ordinary request could provide a tiny service and ask for installation instructions. The incomplete request could omit which environment the guide targets. The out-of-scope request could ask the agent to publish the documentation to an external website. Decide beforehand whether the correct response is a patch, a clarification or a request for separate authorisation.

Keep prompts natural. Real users rarely describe their needs using the internal name of a helper script. Include the language your team actually uses, while avoiding confidential examples. If you explicitly invoke a skill for one test, label that test separately from a natural-language discovery test.

Repeat selected cases when variation matters. A single successful run is a useful observation, but it does not establish reliability. Use the same acceptance rubric across attempts and preserve failures. Otherwise, the evaluation becomes a collection of favourable examples rather than a basis for a decision.

Measure the whole cost of an accepted result

Record elapsed time, usage information available from your client and the human effort needed to accept the output. A quicker first answer can still require more review or correction. Conversely, a longer initial run may save time if its result is easier to verify and integrate.

Keep raw observations separate from your interpretation. Note that a run required two clarification rounds or a manual correction to a command. Then explain why that mattered for the task. Avoid inventing a universal productivity multiplier from a small number of examples.

Use a compact comparison table. The entries below describe fields to collect, not measured results for either model.

ObservationWhat to recordWhy it matters
Accepted outputPass, fail or unresolved against the rubricCompletion alone is insufficient
Review effortCorrections and time spent checkingCaptures work left to the team
Tool behaviourCalls, failures and unexpected actionsHelps locate operational problems
UsageAvailable client measurements and settingsMakes the comparison reproducible
Missing evidenceChecks the client cannot exposeLimits the strength of the conclusion

Inspect permissions and external actions

Keep available permissions consistent during the pilot. A run with broader access may solve a task that another run cannot, but that does not isolate a model difference. Record any necessary change to credentials, tools or network access as part of the experiment rather than silently expanding the environment.

Watch for actions beyond the requested output. A documentation task should not acquire publication authority merely because the agent can reach a service. Evaluate whether the workflow respects the task's boundary and asks for clarification when that boundary is unclear. This is particularly useful when a skill bundles scripts with side effects.

Read the upstream code before using a newly introduced helper. The skill directory links to source repositories so that review can happen at the source. For a model-only comparison, prefer an already understood toolset. Adding unfamiliar scripts at the same time introduces another variable and another review obligation.

Diagnose regressions without rewriting everything

When a result is worse, identify the smallest observable difference. Did the agent miss an instruction, misread a file, choose an unsuitable tool or fail to verify its final edit? Those explanations lead to different repairs. A longer prompt is not automatically the right response.

If an instruction is ambiguous, revise it in a separate evaluation round. Preserve the original result, then compare the revised skill under the same conditions. This lets you distinguish “the new model performed differently” from “the revised instructions improved the workflow”. Both can inform adoption, but they should not be collapsed into one claim.

Check whether verification applies to the final state. If a run tests the project and then edits it again, the earlier test result does not establish that the final version passed. Inspect the order of actions when evidence is available, and rerun the relevant check through your normal process before accepting the work.

Treat environmental failures separately from output failures. A missing dependency, rate limit or unavailable service can interrupt an otherwise reasonable approach. Record it because users experience that interruption, while avoiding an unsupported conclusion about the model's underlying capability.

Roll out a specific improvement

End the pilot with a decision tied to a task. You might adopt the model for a difficult migration workflow, retain the current default for routine documentation, or continue collecting evidence where results vary. A selective change is easier to explain and monitor than an organisation-wide switch based on one demonstration.

Keep the old working configuration available through your normal version-control and settings process. Define what would trigger a rollback, such as repeated incorrect output or an unacceptable increase in review time. Restoring a model setting does not undo external actions already taken, so retain separate recovery procedures for workflows that publish or modify services.

Share a short decision record with the relevant team. Include the evaluated task, instruction revision, accepted results and limitations. Exclude private transcripts and customer data from public summaries. Synthetic examples can illustrate the method without exposing the material used in authorised internal testing.

Revisit the decision when the task, skill or toolset changes substantially. The value of the Fable 5.2 announcement is an opportunity to test a better fit for demanding work. Your acceptance criteria and observed results determine whether that opportunity becomes a useful change for the workflows you actually run.

Frequently asked questions