Meta-Harness Evolver#

What This Skill Does#

Implements the Meta-Harness paper's outer-loop optimization for Hoss — your OpenClaw agent. Each night at 3 AM CDT, this skill:

1. Reads Hoss's current workspace configs + all prior evolution logs
2. Proposes a targeted harness modification via a coding-agent sub-agent
3. Evaluates the proposed harness against a benchmark of ~20 diverse task scenarios
4. Logs the candidate harness + scores + execution traces to the evolution filesystem
5. Posts a summary report to #research Discord channel

The Meta-Harness Loop#

text

Proposer Agent ──(filesystem access)──► Hoss Workspace
      ▲                                   │
      │                          propose harness
      │                                   ▼
      │                          Evaluate on benchmark
      │                                   ▼
log ───┴── store: code + scores + traces ──► ~/hoss-evolution/

Quick Start#

Cron Schedule#

• 3 AM CDT daily — configured via openclaw cron
• Cron command: SKILL=meta-harness-evolution TASK=run_evolution openclaw run

Manual Trigger#

text

/openclaw run --skill meta-harness-evolver --task run_evolution

Directory Structure#

text

~/hoss-evolution/
├── best/                  # Best harness found so far
│   └── current/
├── candidates/            # All evaluated harnesses
│   └── candidate_N/       # One dir per candidate
│       ├── harness/      # The proposed config files (SOUL.md, etc.)
│       ├── eval_scores.json
│       └── traces/        # Execution traces
├── benchmark/             # Evaluation tasks + scorer
│   └── scenarios/         # ~20 diverse task scenarios
├── proposer/              # Proposer's workspace
│   └── logs/              # Proposer's own reasoning traces
└── evolution_log.jsonl    # Full run history

What Can Be Evolved#

Hoss's "harness" = the configs that wrap the LLM brain:

FileWhat It ControlsSOUL.mdCore identity, personality, decision-making styleIDENTITY.mdRole, voice, tone, signature patternsAGENTS.mdSub-agent architecture, coordination protocolTOOLS.mdTool configurations, credentials, key hostsMEMORY.mdLong-term memory structure, what to persistHEARTBEAT.mdActive hours, check priorities, alert thresholds

Constraints (do NOT modify):

• Credentials, API keys, or secrets in TOOLS.md
• Git safety rules (NEVER mutate git config from ~/flume/)
• Security-sensitive groupPolicy settings

The Evolution Algorithm#

1. Seed: Start with Hoss's current configs as iteration 0
2. Propose: Sub-agent reads full history from ~/hoss-evolution/candidates/, identifies failure patterns, proposes 1-2 targeted edits
3. Validate: Lightweight import/syntax check before running full benchmark
4. Evaluate: Run proposed harness against all 20 benchmark scenarios, score each
5. Log: Store candidate harness + scores + proposer reasoning traces
6. Select: Pareto frontier over (performance, simplicity) — proposer decides which candidates to keep exploring from
7. Repeat: Next night's proposer can read ALL prior candidates to build on good ideas

Key Insight from the Paper#

The skill text is the strongest lever — it steers the proposer. Iterating on the proposer's prompt/role description had more effect than changing iteration count or population size.

The Benchmark#

The benchmark lives at ~/hoss-evolution/benchmark/. See references/benchmark-design.md for how to design scenarios and references/harness-spec.md for the full harness spec.

Default benchmark has 20 scenarios across categories:

• Memory: Recall, update, synthesize from memory files
• Code: Write, review, debug code tasks
• Coordination: Spawn sub-agents, synthesize results
• Research: Web search, fetch, summarize, synthesize
• Communication: Draft emails, Discord messages, iMessages
• Quality: Spot errors, inconsistencies, broken links

Each scenario has:

• A concrete task description
• Expected outcome criteria
• A scoring rubric (0-3 per scenario: fail / partial / pass / excellent)

The Proposer Agent#

The proposer is a coding-agent sub-agent (default: coder) that:

• Reads all prior candidates from ~/hoss-evolution/candidates/ via filesystem ops
• Identifies patterns in failed/succeeded candidates
• Proposes targeted, specific edits (NOT wholesale rewrites)
• Writes proposed configs to the new candidate directory
• Logs its reasoning trace so future iterations can build on it

Proposer Skill (passed to sub-agent)#

The proposer's role is defined by the task prompt in scripts/propose_harness.py. Key constraints:

• Can only propose edits to files in the harness spec (SOUL.md, IDENTITY.md, AGENTS.md, TOOLS.md, MEMORY.md, HEARTBEAT.md)
• Must pass lightweight validation before full evaluation
• Should prefer targeted edits over full rewrites
• Must log reasoning trace to proposer/logs/

Workflow Steps#

Step 1: Read Prior Candidates#

bash

# List all prior candidates
ls ~/hoss-evolution/candidates/

# Read best candidate
cat ~/hoss-evolution/best/current/eval_scores.json

# Read history log
tail -20 ~/hoss-evolution/evolution_log.jsonl

Step 2: Run Proposer#

bash

# The sub-agent proposer reads ~/hoss-evolution/ and proposes
# This is triggered by openclaw run with this skill loaded

Step 3: Validate Before Benchmark#

bash

# Quick syntax check
bash ~/hoss-evolution/scripts/validate.sh <candidate_dir>

Step 4: Run Benchmark#

bash

# Evaluate candidate against all 20 scenarios
python3 ~/hoss-evolution/scripts/evaluate.py <candidate_dir>

Step 5: Log Results#

bash

# Scores + traces written to candidate dir automatically
# Evolution log updated

Step 6: Post to Discord#

bash

# Posts summary to #research
python3 ~/hoss-evolution/scripts/post_to_research.py <candidate_dir>

Scoring#

Final score = weighted average across scenarios:

• Memory tasks: 25%
• Code tasks: 25%
• Coordination: 15%
• Research: 20%
• Communication: 10%
• Quality: 5%

Results are tracked as a Pareto frontier: for each candidate, log both score and "complexity" (size/diff of changes). Simpler harnesses that score equally get priority.

Resources#

• references/harness-spec.md — Full spec of what constitutes Hoss's harness, what can/cannot be modified
• references/benchmark-design.md — How to design benchmark scenarios, scoring rubrics, how to add new scenarios
• references/evolution-logic.md — Detailed evolution algorithm, parent selection, Pareto frontier logic
• scripts/run_evolution.py — Main entry point, runs the full loop
• scripts/propose_harness.py — The proposer sub-agent task definition
• scripts/evaluate.py — Benchmark runner
• scripts/post_to_research.py — Discord reporter

Notes#

• The proposer sub-agent runs with runtime=subagent, not ACP — it needs filesystem access to ~/hoss-evolution/
• Cron is configured outside this skill via openclaw cron
• If the proposer fails to produce a valid candidate, the iteration is skipped (no penalty)
• Benchmark scenarios should be diverse enough that no single strategy can game all of them
• The evolution workspace is NOT inside ~/.openclaw/ — it's at ~/hoss-evolution/ to keep it separate from operational configs