
Harness Learn
FreeOptimize harness genomes with automated learning cycles.
Free · Opens the source repo
What Harness Learn does
Harness Learn is a tool designed to facilitate the optimization of harness policy genomes through an automated learning cycle. By utilizing the metaharness learn command, users can evolve their harness policies against a scored task corpus, eliminating the need for manual prompt adjustments. The tool is particularly beneficial for developers and data scientists who are looking to enhance the performance of their AI models by leveraging a systematic approach to learning and evaluation.
The skill operates by running a GEPA learning cycle, which allows for the evaluation of multiple candidate policies based on their performance on held-out slices. This process ensures that only the most effective policies are promoted, thereby improving the overall performance of the harness. Users can initiate a learning run with a simple command, and the default dry-run option allows for cost estimation without incurring any charges. This feature is crucial for budgeting and planning, especially in environments where resource management is key.
To use Harness Learn, a local clone of the metaharness repository is required. This setup ensures that the learning harness has access to the necessary resources and configurations. The tool gracefully handles cases where the repository is not available, providing clear instructions for users to follow. This design choice minimizes disruptions and allows users to focus on their learning objectives without unnecessary complications.
In summary, Harness Learn is an essential tool for those involved in AI development who seek a structured and efficient way to improve their harness policies. Its automated learning cycles and cost-effective dry-run feature make it a valuable addition to any developer's toolkit.
When to use it
Use this skill when you want to improve the performance of a harness policy on specific tasks without manual intervention.
When not to use it
This tool is not suitable if you do not have access to the metaharness repository or if you prefer manual prompt adjustments over automated learning cycles.
What you can build with it
Improving AI Model Performance
When a harness's policy prompt is underperforming, use Harness Learn to run a learning cycle that systematically improves the policy based on scored tasks.
Cost Estimation for Learning Runs
Before committing to a learning run, utilize the dry-run feature to assess costs and ensure budget alignment.
Inspecting Promoted Policies
After a learning run promotes a genome, pair the skill with `harness-gepa --op render` to analyze the details of the newly promoted policy.
How to install Harness Learn
View source1. Install with the skills CLI
npx skills add ruvnet/ruflo/harness-learn --agent claude-code2. Or install it manually
Download the skill folder and drop it into ~/.claude/skills/ for all projects, or .claude/skills/ to scope it to one repo. Restart Claude Code so it picks up the new skill.
Anthropic's agentic coding CLI, and the reference implementation of Agent Skills. Drop a skill folder into ~/.claude/skills and Claude Code loads it automatically whenever a task matches the skill's description. Claude Code docs
Inside SKILL.md
Written by ruvnetSurfaces metaharness learn — the upstream GEPA learning harness that
evolves harness policy genomes against a scored task corpus instead of
hand-editing prompts. Candidates are scored on held-out slices and only
measured winners promote (the shipped cand-6 genome is the first such
promotion: holdout gold 2/12 → 3/12, zero regressions).
When to use
- A harness's policy prompt underperforms on a task family and you want a measured improvement loop rather than manual prompt iteration.
- Pricing a learning run before committing spend — the default dry-run resolves the slice manifest and reports cost without any model calls.
- After a learn run promotes a genome: pair with
harness-gepa --op renderto inspect what the promoted policy actually says.
Preconditions (upstream design)
The learning harness (GEPA + SWE-bench + Docker) is too heavy for the npm
package, so learn needs a local clone:
git clone https://github.com/ruvnet/metaharness.git
node scripts/learn.mjs --repo ./metaharness --host claude-code --model haiku --slice slices/lite.json
Without a checkout the script emits {status: "checkout-required"} and
exits 0 — a precondition report, not an error (distinct from
degraded: true, which means the npm package itself is absent). The
managed-service path (gateway-side learn jobs, no checkout) is upstream's
ADR-235 follow-up and not available yet.
Algorithm
Implementation: scripts/learn.mjs.
- Validate
--repoexists when given; export it as$METAHARNESS_REPO. - Invoke the pinned
metaharnessbinary (metaharness@~0.3.0, local install or one-time versioned cache — never@latest):metaharness learn --host <h> --model <m> --slice <s> [--run]via_harness.mjs(graceful degradation, hard timeout). - Default timeouts: 120s dry-run, 600s with
--run— real runs on larger slices need an explicit--timeout-msmatched to slice size × model cost. - Detect the checkout-required message → structured payload, exit 0.
- Parse the trailing JSON report when upstream emits one; otherwise return
the raw report text under
rawReport.
Cost note
--run is the ONLY path that spends. Everything else — dry-run, checkout
probe, degraded path — is $0. The MCP tool (metaharness_learn) has a 120s
subprocess budget; run real learning cycles from a terminal via
ruflo metaharness learn ... --run --timeout-ms <big>.
Exit codes
0— report produced (or dry-run, checkout-required, degraded)1—--alert-on-failand the learn run reported failure2— config error (bad--repopath)
Frequently asked questions about Harness Learn
Similar skills
Arize Evaluator
Streamline LLM evaluation workflows on Arize.
Troubleshoot
Analyze logs to understand chat agent behavior.
Agentic Evaluation
Enhance AI outputs through iterative evaluation and refinement.
RAG Evaluation
Evaluate retrieval-augmented generation benchmarks efficiently.
NV-Reason-CXR
Run smoke tests for chest X-ray reasoning models.
Clinical ASR Evaluation
Score and evaluate clinical ASR manifests effectively.
