Ratchet

Ratchet is a self-evolving single-agent system that turns its own failures into
a governed library of natural-language skills, so a frozen LLM keeps improving
on code and patch benchmarks without weight updates. It is the reference
implementation of
Ratchet: A Minimal Hygiene Recipe for Self-Evolving LLM Agents,
whose contribution is the library's lifecycle discipline: skills are
synthesised from clustered failures, injected as prompt guidance, and retired
once they stop helping, under a bound that keeps the library from drifting.

Ratchet assumes a reliable metric already grades each attempt. Where none
exists, Self-Evolving-Agents-Double-Ratchet
co-evolves the metric alongside this skill loop and pins a copy of this
repository as its skill loop.

Each task: route through an LLM with 0 or 1 skill injected as prompt
guidance, log the attempt, judge it with a verbal critic. Each round:
synthesise new skills from clustered failures; retire ones that don't
help. A separate bank of meta-skills refines how skills are written.
Nothing is deleted — retired skills keep their evidence for rollback.

Papers

This repository is the reference implementation for two papers:

• Ratchet: A Minimal Hygiene Recipe for Self-Evolving LLM Agents —
  arXiv:2605.22148. The technical
  paper linked above: the full method, the non-divergence guarantee, and
  the ablations.
• Library Drift: Diagnosing and Fixing a Silent Failure Mode in
  Self-Evolving LLM Skill Libraries —
  arXiv:2605.19576. Accepted to the
  ICML 2026 Workshop on Failure Modes in Agentic AI (FAGEN@ICML 2026),
  Seoul, South Korea.

See Reproducibility for the reported configuration
and how to rerun it.

Quickstart

Prerequisites

• Python 3.10+
• An AWS account with Bedrock access enabled in your region for:
  • anthropic.claude-opus-4-7 (and claude-sonnet-4-6 as fallback)
  • cohere.embed-v4
• (SWE-Bench only) Docker daemon running

1. Install

git clone https://github.com/amazon-science/Self-Evolving-Agents-Ratchet
cd Self-Evolving-Agents-Ratchet
pip install -e '.[dev,bedrock,mbpp]'
# add ',swebench' if you plan to use the SWE-Bench Verified suite

2. Set credentials

export AWS_REGION=us-east-1
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
# Optional but recommended (avoids HuggingFace rate limits):
export HF_TOKEN=...
# Optional model overrides:
# export SKILLEVO_BEDROCK_MODEL_ID=global.anthropic.claude-opus-4-7
# export SKILLEVO_BEDROCK_FALLBACK_MODEL_ID=global.anthropic.claude-sonnet-4-6
# export SKILLEVO_BEDROCK_EMBED_MODEL_ID=global.cohere.embed-v4:0

3. Verify the environment

skillevo doctor                          # <30s pre-flight (creds, Bedrock, embed)
skillevo doctor --suite swebench         # + Docker / git
python scripts/run_skill_loop.py mbpp --n-train 3 --n-eval 2 --rounds 1 --tag smoke

If checks fail, see Troubleshooting.

4. Real run on the hard subset

Subset files are generated locally, not shipped: run baseline_mbpp.py
then select_hard_subset.py (see Run) to write
results/subsets/<suite>_hard<N>_t<T>_e<E>_seed<S>.json, then:

# MBPP+ hard-100 (60 train / 40 eval)
python scripts/run_skill_loop.py mbpp --rounds 100 \
  --subset results/subsets/mbpp_hard100_t60_e40_seed42.json

# SWE-Bench Verified hard-150 (90 train / 60 eval) — same pattern

Selection is deterministic given the same base model, seed, and probe
count, so the same command reproduces the same subset.

5. Inspect

RUN=$(ls -td results/mbpp/mbpp_r* | head -1)
python scripts/plot_skill_loop.py "$RUN/loop.json"
skillevo --db "$RUN" skill list
skillevo --db "$RUN" audit --limit 20

Bringing your own skill bank

If you already have skills authored some other way, import them
as YAML matching skillevo/skill/schema.py (id / intent / signals_match /
guidance.{applies_when, key_insight, common_pitfalls, verify_before_returning}).
Drop the files into the run dir's skills/ folder, then load:

skillevo --db results/<suite>/<run-id> skill load

Two flags support this case:

• do_not_retire: true in any skill YAML — exempts that skill
  from contribution-score retirement and bank-cap eviction. Use for
  compliance / business-required skills you must keep regardless of
  measured contribution. The curator still emits a report with the
  live score so you can review unlock candidates.
• --no-synth — disables per-round skill synthesis. Capsules,
  verdicts, stats, and curator still run, so you get a full audit of
  the existing bank without any new skills being authored.

Recommended phased rollout:

1. Audit (read-only): --no-synth + every skill do_not_retire: true.
   You get per-skill contribution-score reports without changing the bank.
2. Curated retirement: drop --no-synth (still no new skills) but
   selectively unlock skills the audit flagged as low-performing.
3. Full self-evolution: drop the locks. Synth + curator manage the
   bank end-to-end.

Missing fields are tolerated. signals_match, tags, preconditions
all default to empty lists; the retriever drops empty fields from its
scoring surface and still matches on intent, name, and description.
You should still set intent (one short imperative sentence:
"Sum a list of numbers") and description (one line describing
when the skill applies) — without those two, the retriever has nothing
to score against and the skill becomes effectively un-routable.

Bringing your own task

Custom suites plug in via skillevo.adapters — no fork. See
examples/custom_task/ for a worked example
(CSV/JSON record extraction, ~150 lines).

from skillevo.adapters import BenchResult, register

class MyAdapter:
    name = "my_suite"
    def load(self, n_train, n_eval, seed, **kw): ...   # → (train, eval) lists
    def run(self, engine, capsules, tasks, **kw): ...  # → BenchResult
    def pack_cli_kwargs(self, args): return {}         # forward CLI flags

register("my_suite", MyAdapter)

export SKILLEVO_ADAPTERS=path/to/my_adapter.py
python scripts/run_skill_loop.py my_suite --rounds 5

Set tags=["my_suite", "python_code"] on each task to reuse the
code-gen prompt. For non-code outputs, add a template to
skillevo/llm/prompts.py and a branch in skillevo/engine/solver.py.

Architecture at a glance

  ┌──────────────────── per task ────────────────────┐
  │                                                  │
  │   task  ──▶  Router  ──▶  Solver  ──▶  Capsule   │
  │                │           │            │        │
  │            LLM pick     LLM write     SQLite     │
  │           (0 or 1       code /        row        │
  │            skill)       patch                    │
  │                                         │        │
  │                                         ▼        │
  │                                      Critic      │
  │                                      LLM verdict │
  │                                         │        │
  │                                         ▼        │
  │                                      Verdict     │
  │                                      SQLite row  │
  └──────────────────────────────────────────────────┘

  ┌─────────────── end of round / cadence ───────────┐
  │                                                  │
  │   Synthesizer   ──▶   new Skill (.yaml)          │
  │   (reads Verdicts,     ACTIVE, in bank           │
  │    capsules,                                     │
  │    active meta-skill,                            │
  │    active bank)                                  │
  │                                                  │
  │   MetaSynthesizer ──▶ new MetaSkill (.md)        │
  │   (every N rounds     retires prior same-suite   │
  │    or M tasks)         active                    │
  │                                                  │
  │   Curator       ──▶  retire Skills whose         │
  │                       contribution ≤ -0.10       │
  │                       (row + evidence stay)      │
  └──────────────────────────────────────────────────┘

Key artifacts

Skill

A skill is not executable code — it is natural-language guidance
injected into the solver's LLM prompt. The schema (fixed in the
synthesize_skill prompt) is:

id: sliding_window
name: sliding_window
version: 0.1.0
description: <one-liner>
intent: <imperative match phrase>
signals_match: [sliding_window, fixed_length_span]
tags: [auto]
status: active
author: synthesized
guidance:
  applies_when: tasks over a fixed-size contiguous span
  key_insight: maintain a running window; slide by one on each step
  common_pitfalls:
    - off-by-one on the window edge
    - forgetting to shrink on mismatch
  verify_before_returning: window length stays constant; start <= end

• Lifecycle: ship ACTIVE; retire when
  total_tasks_since_added ≥ 100 AND
  contribution ≤ −0.10, where
  contribution = (successes − failures) / total_tasks_since_added.
  Retired skills become DEPRECATED; never deleted.
• Bank cap skill_bank_max = 100: when full, lowest-contribution
  ACTIVE is demoted to make room for a new synth.
• Dedup on synth: cosine ≥ 0.85 between full-YAML embeddings
  re-prompts the LLM with the duplicate's id as feedback.

MetaSkill

Authoring advice consumed by the Synthesizer (not the Skill schema —
that's hard-coded). One ACTIVE meta-skill per suite at a time.
Defaults live at examples/meta_skills/default_{mbpp,swebench}.md
and get copied into the run's bank on first call.

Auto-refresh is pinned off (cfg.metaskill_refresh_every_rounds = 1_000_000):
it consumes a substantial share of a run's wall time for little measured
benefit. Refresh manually via skillevo synth metaskills or
--meta-every N.

Per-task flow (Engine.run(task))

1. Router (router/router.py) — bank ≤ 20: every ACTIVE skill →
   LLM. Larger: tf-idf top-10 ∪ embed top-10 → LLM. Returns skill id
   or NONE.
2. Solver (engine/solver.py) — solve_code / solve_patch by
   task tags; injects the chosen skill's guidance into the prompt.
3. Grader — subprocess (MBPP) or Docker harness (SWE-Bench).
4. Capsule — one SQLite row per attempt with the full forensic
   payload.
5. Critic (engine/critic.py, end of round) — one LLM call per
   train capsule, writes a Verdict.

Per-round flow (what run_skill_loop.py does each round)

eval   ──▶  adapter.run over eval set (ACTIVE-only routing)
train  ──▶  adapter.run over train set (ACTIVE routing; fresh, uncached)
critic ──▶  one Verdict per train capsule
[every N rounds or M tasks:]
  metasynth ──▶ new MetaSkill for this suite (retires prior same-suite)
synth   ──▶  cluster recent train failures → new Skill(s) (ACTIVE)
curate  ──▶  retire Skills whose contribution ≤ -0.10 (keep row + evidence)
snapshot──▶  full bank dump (yaml) + DB checkpoint

Auto-rollback (persistence-gated): a round's eval pass@1 must regress
by more than --rollback-threshold (default 0.10) below best-so-far
for --rollback-consecutive rounds in a row (default 5) before the
bank snapshot of the best round is restored and synth/curate skip
that round. A single recovery resets the counter — this is a
persistence gate, not a trend check.

LLM calls, summarised

Models: global.anthropic.claude-opus-4-7 (primary, falls back to
claude-sonnet-4-6) + global.cohere.embed-v4:0. Override via
SKILLEVO_BEDROCK_MODEL_ID / SKILLEVO_BEDROCK_FALLBACK_MODEL_ID /
SKILLEVO_BEDROCK_EMBED_MODEL_ID. Bedrock is the only backend.

Install — extras

.[dev,bedrock] (core + Bedrock), .[mbpp] (datasets / evalplus),
.[swebench] (datasets / SWE-bench harness / Docker). AWS perms
needed: bedrock-runtime:Converse and bedrock-runtime:InvokeModel
on the model ids above.

Run

Built-in suites:

Direct-LLM baselines + hard-subset selection (one-shot)

Before training, establish the "LLM alone" floor (5 calls/task, no
skills) and pick the hard subset. Both scripts are resumable. This
step is required before any --subset run: subset files are produced
here, not shipped in the repo.

mkdir -p logs results/baselines
nohup python scripts/baseline_mbpp.py     --out results/baselines/mbpp.jsonl     > logs/baseline_mbpp.log 2>&1 &
nohup python scripts/baseline_swebench.py --out results/baselines/swebench.jsonl --work-dir results/baselines/swebench_workdir > logs/baseline_swebench.log 2>&1 &

python scripts/select_hard_subset.py --source results/baselines/mbpp.jsonl --suite mbpp     --n-hard 100 --n-train 60 --n-eval 40
python scripts/select_hard_subset.py --source results/baselines/swebench.jsonl --suite swebench --n-hard 150 --n-train 90 --n-eval 60

Training loops

# MBPP+ on the paper subset
python scripts/run_skill_loop.py mbpp --rounds 100 \
  --subset results/subsets/mbpp_hard100_t60_e40_seed42.json

# SWE-Bench Verified — needs Docker daemon
python scripts/run_skill_loop.py swebench --rounds 20 \
  --subset results/subsets/swebench_hard150_t90_e60_seed42.json

# Random slice instead (no subset)
python scripts/run_skill_loop.py mbpp --n-train 150 --n-eval 100 --rounds 10

Common flags (scripts/run_skill_loop.py --help)

Run output

Run dir: results/<suite>/<suite>_r<R>_t<N>_e<M>_<UTCts>[_<tag>]/.
Under --db-mode=per-run (default) it's self-contained:

config.json   train_ids.json   eval_ids.json   loop.json   run.log
history.db                  full SQLite (capsules + verdicts + skills + audit)
skills/                     skill YAMLs
meta-skills/                meta-skill .md files
checkpoints/                per-round bank snapshots
solver_cache/               eval LLM cache
round_NN/
  eval.json / train.json    BenchResult (total, resolved, pass@1)
  *_capsules.jsonl          one capsule per line
  synth_skills.json         created + evicted + skipped IDs
  synth_metaskills.json     only on rounds where meta cadence fired
  curate.json               per-skill decisions + contribution score
  bank_{skills,metaskills}.yaml

Plot:

python scripts/plot_skill_loop.py "$(ls -td results/mbpp/mbpp_r* | head -1)/loop.json"
# → loop_curve.png + loop_summary.md

Data model

All types in skillevo/core/types.py:

Nothing is deleted during a run. Prune capsules explicitly with
skillevo capsule prune --keep-days 90.

Layout

skillevo/
  adapters/      suite registry (built-ins + plug-in API)
  core/          types, config, exceptions
  skill/         metaskill/  capsule/  aggregator/  curator/
  router/        engine/     synthesizer/  evaluator/
  storage/       llm/        observability/  cli/

results/                   runtime output
  bank.db                  global fallback (`skillevo --db ...` for per-run)
  <suite>/                 per-suite root, contains per-run dirs (see Run output)

examples/
  custom_task/             worked plug-in example (CSV/JSON extraction)
  skills/   meta_skills/   seed skills + default meta-skills
tests/                     unit + integration

CLI

skillevo doctor [--suite swebench]               # pre-flight env check

# Training runs write per-run DBs — point `--db` at a run dir or a
# history.db file. Without --db the CLI reads results/bank.db.
RUN=$(ls -td results/mbpp/mbpp_r* | head -1)
skillevo --db "$RUN" skill list                  # ACTIVE + DEPRECATED
skillevo --db "$RUN" meta list
skillevo --db "$RUN" audit --limit 20
skillevo --db "$RUN" curate                      # re-run retirement
skillevo --db "$RUN" capsule prune --keep-days 90
skillevo solver-cache list                       # no --db needed

Reproducibility

This repository contains the full implementation used for the
papers above: the skill loop, all five roles, the governance
mechanisms (outcome-driven retirement, bounded active-cap, meta-skill
authoring prior, canonicalisation, rollback), the ablation harness, and
the evaluation adapters for MBPP+ and SWE-bench Verified.

What is included. Source code, the ablation runner
(scripts/run_ablations.py), the subset-selection script, default
hyperparameters, and a synthetic worked example
(examples/custom_task/) that runs without any benchmark download.

What is not included. No run artifacts, logs, databases, model
outputs, or result files are distributed. results/ and logs/ are
git-ignored by design; every artifact is regenerated locally by the
commands above. The skill and meta-skill YAML files in examples/ are
illustrative placeholders, not the banks evolved during the paper runs.

Reproducing a paper run. The reported configuration is: MBPP+
hard-100 (60 train / 40 eval) for 100 rounds, and SWE-bench Verified
hard-150 (90 train / 60 eval) for 20 rounds, each over seeds 42, 7, and
13, with --skill-retire-contribution-threshold -0.10,
--skill-retire-min-tasks 100, --skill-bank-max 50,
--rollback-threshold 0.10, and --rollback-consecutive 5. Run
baseline_mbpp.py and
select_hard_subset.py first to build the subset, then
run_skill_loop.py per the Training loops section,
then scripts/run_ablations.py for the A1–A8 arms.

Expected variance. Exact numbers will not reproduce bit-for-bit.
The loop calls a hosted LLM whose sampling is non-deterministic and
whose served checkpoint may change over time, and the router, critic,
synthesizer, and curator are all LLM calls. Seeds fix task selection,
splits, and clustering order, not model output. Report multi-seed means
with dispersion rather than single-run values, and expect the direction
and rough magnitude of the effect to reproduce, not the exact digits.

Environment. Python 3.10+, AWS Bedrock for all model calls
(anthropic.claude-opus-4-7, claude-sonnet-4-6, cohere.embed-v4),
and Docker for the SWE-bench harness. Pinned dependency ranges are in
pyproject.toml. skillevo doctor verifies the environment in under
30 seconds before a long run.

Tests

pytest --ignore=tests/integration/test_capsule_replay.py

All tests run offline — tests/conftest.py swaps the LLM + Embedder
singletons for StubLLMClient + HashEmbedder. Production code has
no stub path; Bedrock is the only backend.

The three SWE-bench capsule-replay integration tests in
tests/integration/test_capsule_replay.py exercise the SWE-bench harness
and expect its environment; skip them (as above) for a quick offline check.

What's NOT here

• No model training. Skills are natural-language guidance, not
  fine-tuned weights.
• No executable skill code. Skills never run — they decorate the
  solver's prompt, and the solver's LLM writes the actual code.
• No routing DSL. The router is an LLM call; meta-skills don't
  participate in routing.
• No pre-training / seed skills required. An empty
  results/<suite>/skills/ dir is the correct starting state; synth
  fills it.

Troubleshooting

Citation

@misc{ratchet2026,
  title  = {Ratchet: A Minimal Hygiene Recipe for Self-Evolving LLM Agents},
  author = {Xing Zhang and Yanwei Cui and Guanghui Wang and Ziyuan Li and
            Wei Qiu and Bing Zhu and Peiyang He},
  year   = {2026},
  eprint = {2605.22148},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2605.22148}
}

@misc{librarydrift2026,
  title  = {Library Drift: Diagnosing and Fixing a Silent Failure Mode
            in Self-Evolving LLM Skill Libraries},
  author = {Xing Zhang and Yanwei Cui and Guanghui Wang and Ziyuan Li and
            Wei Qiu and Bing Zhu and Peiyang He},
  year   = {2026},
  eprint = {2605.19576},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2605.19576},
  note   = {ICML 2026 Workshop on Failure Modes in Agentic AI (FAGEN@ICML 2026)}
}

Contributing

See CONTRIBUTING for guidelines, and
CODE_OF_CONDUCT for the code of conduct this
project has adopted.

Security

See CONTRIBUTING for more information.

License

This project is licensed under the Apache-2.0 License.