An agent memory and learning system that replaces naive file-based memory with a structured pipeline: real-time capture, observable Candidate → Learning → Promotion queues, automated cron-based audit, and continuous promotion of lessons into skills and agent instructions.
The system runs as four layers:
Candidate → Learning → Promotion → Done using scripts/memory-pipeline.py, with learning/pipeline/candidates.jsonl, promotion-queue.json, status.json, and dashboard.md making backlog and coverage visible.sessions_history only when it is verified in the runtime. The jobs detect missed lessons and maintain lifecycle (HOT → WARM → COLD).memory/YYYY-MM-DD.md digest records decisions, paths, risks, links, and follow-ups; only reusable lessons are extracted into SQLite.learning/ SQLite → skills/ SKILL.md → the 7 root Markdown control-plane files ( AGENTS.md, HEARTBEAT.md, IDENTITY.md, MEMORY.md, SOUL.md, TOOLS.md, USER.md). The full 3+7 system co-evolves.Author: Rockway Chen · rockwaychen@gmail.com · GitHub: LingmaFuture
clawhub install is only step 1#A raw ClawHub install only copies the skill files. It does not automatically wire the agent into a self-improving operating loop.
Use this maturity ladder:
LevelWhat is configuredResult1/5clawhub install self-improving-compoundFiles are present; almost no behavior changes yet.2/5learning/ initialized and CLI verifiedManual logging/search works.3/5Capture gate added to agent instructionsThe agent remembers to log lessons before final replies.4/5Cron jobs installed and delivery configuredMissed lessons, failures, daily memory, and steward checks run automatically.5/5Hooks/env/collector verifiedActivation reminders, error capture, path resolution, and daily factual memory are reliable.
You must distinguish two roots:
bash
# Workspace root: where memory/, learning/, AGENTS.md, etc. live
export OPENCLAW_WORKSPACE="/path/to/workspace"
# Optional shared lesson store for multiple workspace roots
# export SELF_IMPROVING_LEARNING_ROOT="$HOME/.openclaw/shared-learning"
# Skill root: where this skill was installed
export SELF_IMPROVING_SKILL_DIR="$OPENCLAW_WORKSPACE/skills/self-improving-compound"
export SELF_IMPROVING_LEARNINGS_CLI="$SELF_IMPROVING_SKILL_DIR/scripts/learnings.py"
For OpenClaw's default workspace this is usually:
bash
export OPENCLAW_WORKSPACE="$HOME/.openclaw/workspace"
export SELF_IMPROVING_SKILL_DIR="$OPENCLAW_WORKSPACE/skills/self-improving-compound"
export SELF_IMPROVING_LEARNINGS_CLI="$SELF_IMPROVING_SKILL_DIR/scripts/learnings.py"
Do not write durable learnings into the skill directory. By default learning/ belongs under the workspace root; use SELF_IMPROVING_LEARNING_ROOT or --learning-root only when several workspaces should share one lesson store.
bash
clawhub install self-improving-compound
cd "$SELF_IMPROVING_SKILL_DIR"
chmod +x scripts/*.py scripts/*.sh hooks/*.sh 2>/dev/null || true
If the skill is copied manually, set SELF_IMPROVING_SKILL_DIR to the copied directory.
The bundled shell helpers require bash. On POSIX sh-only hosts, use the Python CLI directly and skip the .sh helpers.
bash
python3 "$SELF_IMPROVING_LEARNINGS_CLI" --root "$OPENCLAW_WORKSPACE" init
python3 "$SELF_IMPROVING_LEARNINGS_CLI" --root "$OPENCLAW_WORKSPACE" status
python3 "$SELF_IMPROVING_LEARNINGS_CLI" --root "$OPENCLAW_WORKSPACE" search "install smoke test" --limit 3
Expected result:
learning/memory_tree/chunks.db exists.status exits successfully.search works even when there are no results.Add a compact rule to the agent's durable instruction file, usually AGENTS.md:
markdown
## Self-improvement capture gate
Before the final reply after any non-trivial task, check whether the work involved a user correction, non-obvious failure, tool/API quirk, workaround, format mismatch, missing capability, or reusable convention.
If yes:
1. Search existing learnings first:
`python3 $SELF_IMPROVING_LEARNINGS_CLI --root $OPENCLAW_WORKSPACE search "<keywords>" --limit 5`
2. If no suitable entry exists, log the durable lesson with `log-correction`, `log-error`, `log-learning`, or `log-feature`.
3. Keep entries compact, prevention-oriented, and secret-free.
Without this phase the skill remains mostly passive; the agent will not consistently capture lessons in real time.
Cron installation is not automatic. The templates live in scripts/setup-cron.json; agent-facing instructions live in scripts/setup-cron-agent.md.
Recommended OpenClaw flow:
scripts/setup-cron.json. Check existing jobs first and update instead of duplicating."cron list.The pipeline normally includes:
JobPurposeSelf-Improving Light CheckFrequent lightweight scan for missed corrections, errors, and blockers.Learning Audit HeavySystem/cron failure audit, learning-audit.py --log, lifecycle maintenance.Daily Memory DigestWrites memory/YYYY-MM-DD.md factual continuity notes and extracts reusable lessons.Daily Workspace StewardExports learning/, checks skills/, and inspects the 7 root Markdown control-plane files.
Important cron requirements:
schedule.tz to the user's actual timezone.delivery.channel and delivery.to; otherwise reports may not reach the user.cron update for existing jobs; do not create duplicates.sessions_list / sessions_history only after verifying those tools can access the target session from isolated cron.Hooks are optional but improve activation. They are runtime-specific.
Dry-run the bundled hooks first:
bash
"$SELF_IMPROVING_SKILL_DIR/hooks/activator.sh" "install smoke test" || true
"$SELF_IMPROVING_SKILL_DIR/hooks/error-detector.sh" "install" "smoke test" || true
If your client supports command hooks, wire:
hooks/activator.sh as a pre-prompt / prompt-start reminder.hooks/error-detector.sh as a post-error / failed-command reminder.If your runtime has no hook system, skip this phase and rely on the capture gate plus cron. Do not invent config keys; use your runtime's documented hook mechanism.
The bundled scripts/memory-pipeline.py adds an explicit queue and dashboard around cron/context scanning. It is optional for tiny installs but recommended for reliable systems.
bash
export SELF_IMPROVING_MEMORY_PIPELINE="$SELF_IMPROVING_SKILL_DIR/scripts/memory-pipeline.py"
# Optional if the OpenClaw session key is ambiguous:
# export SELF_IMPROVING_MAIN_SESSION_KEY="agent:main:<channel>:direct:<id>"
python3 "$SELF_IMPROVING_MEMORY_PIPELINE" --base "$OPENCLAW_WORKSPACE/learning/pipeline" dashboard
State files:
learning/pipeline/candidates.jsonl — suspected lessons awaiting dedupe/logging.learning/pipeline/promotion-queue.json — logged lessons awaiting skill/root-file promotion.learning/pipeline/cursor.json — last processed transcript line for incremental cron scans.learning/pipeline/dashboard.md and learning/dashboard.md — human-readable health view.Cron Light Check should prefer this flow: collect-incremental → add/mark candidates → log learnings → add/mark promotions → dashboard → commit-cursor. Commit the cursor only after successful processing.
For reliable cron audits, prefer deterministic collectors over implicit chat context. A collector is a local command that reads the runtime's session/transcript store and writes recent visible user/assistant text to a Markdown or JSON file. It should skip tool outputs, hidden thinking, secrets, and raw long transcripts.
Light Check can use a local recent-conversation collector:
bash
export SELF_IMPROVING_LIGHT_CONTEXT_COLLECTOR="python3 /path/to/recent-context-collector.py --limit 60"
Collector contract:
0 only when context was exported successfully.BLOCKED: collector_unavailable instead of claiming success.Daily Memory Digest can also use a local collector if your runtime has one:
bash
export SELF_IMPROVING_DAILY_COLLECTOR="python3 /path/to/collector.py"
bash "$SELF_IMPROVING_SKILL_DIR/scripts/daily-memory.sh" --root "$OPENCLAW_WORKSPACE"
If no collector is configured, the helper prints the target note contract and the agent must gather context with available runtime tools.
Run this checklist after installation:
bash
python3 "$SELF_IMPROVING_LEARNINGS_CLI" --root "$OPENCLAW_WORKSPACE" log-learning \
--summary "Self-improving install smoke test" \
--details "Temporary entry to verify logging path; mark resolved or delete if desired." \
--pattern "install:smoke-test" \
--area "domain:setup"
python3 "$SELF_IMPROVING_LEARNINGS_CLI" --root "$OPENCLAW_WORKSPACE" search "install smoke test" --limit 5
bash "$SELF_IMPROVING_SKILL_DIR/scripts/learning-export.sh"
bash "$SELF_IMPROVING_SKILL_DIR/scripts/daily-memory.sh" --root "$OPENCLAW_WORKSPACE" --date "$(TZ=Asia/Shanghai date +%F)"
Then verify:
learning/memory-export.md exists.learning/status.json exists.memory/YYYY-MM-DD.md target is clear if daily memory is enabled.cron list shows the expected enabled jobs with nextRunAtMs.SymptomLikely causeFixclawhub install succeeded but nothing changesCapture gate and cron not configuredComplete phases 3-4.Cron runs but finds no conversation contextIsolated session has no main history or sessions_history is restrictedConfigure SELF_IMPROVING_LIGHT_CONTEXT_COLLECTOR; use sessions_history only as a verified fallback.Learnings appear under the skill directoryWrong --rootSet OPENCLAW_WORKSPACE; always pass --root.Cron summaries disappeardelivery missing channel/recipientSet delivery.channel + delivery.to.Daily memory is generic or emptyNo collector/context sourceConfigure SELF_IMPROVING_DAILY_COLLECTOR or improve the cron prompt.Duplicate cron jobsSetup re-run without idempotency checkcron list first; update by job name.
The 3 state directories are:
memory/ — factual daily continuity.learning/ — SQLite-backed execution lessons.skills/ — hardened reusable procedures.The 7 root Markdown control-plane files are:
AGENTS.md — workspace contract, routing, execution policy, safety boundaries.HEARTBEAT.md — lightweight check-in surface; often intentionally empty when cron owns timing.IDENTITY.md — compatibility pointer or short identity bridge.MEMORY.md — pinned long-term hot context.SOUL.md — agent identity/persona.TOOLS.md — concrete local environment/tool facts.USER.md — durable user profile and collaboration preferences.The steward loop should keep these layers aligned with small, safe edits only. It must not rewrite persona, weaken safety/privacy rules, or promote volatile daily facts into root-level bloat.
Use this system for durable improvement, not for every bump in the road.
Before a final reply, run this quick check:
If yes, search existing learnings first, then log the lesson before replying. Do not rely on a “mental note.”
A good entry usually has at least one of these properties:
Do not log routine noise such as obvious typos, expected validation failures, or errors that were solved immediately with no transferable lesson.
Not all lessons go to the same place. Route based on type:
Lesson typeDestinationExampleUser facts, preferences, system stateMEMORY.md / memory/YYYY-MM-DD.md"Rockway prefers newspaper theme"Execution mistakes, tool gotchas, workaroundslearning/ SQLite"Python shadowing broke promote"Stable rules, workflows, anti-patterns discoveredowning skills/<skill>/SKILL.md"cron isolation means no session context"Behavioral constraintsAGENTS.md"Don't commit workspace root"Environment-specific tool knowledgeTOOLS.md"Tailscale node name"
The full 3+7 system co-evolves: fixing one layer while leaving another stale is half-done work. When a lesson reveals a skill is stale, upgrade it immediately and bump its version.
This skill merges three design lineages into one portable package:
LineageRoleWhat We Keptactual-self-improvementExecution corePython CLI ( scripts/learnings.py), structured logging, JSON evals, search-before-log dedupeOpenHuman memory-treeStorage coreSQLite chunks, FTS search, entity index, scores, hotness, async jobs, deterministic tree buffers, lifecycle status, idempotent ingestself-improving-compoundMemory architectureHOT/WARM/COLD lifecycle, workspace-scoped learning/, lightweight bootstrap markdownself-improving-agent-localPromotion & hooksQuantified promotion thresholds, OpenClaw hook guidance, pattern-key recurrence rules
learning/#text
learning/
├── memory_tree/chunks.db # SQLite source of truth for durable learnings
├── index.md # SQLite-generated snapshot (entries, lifecycle, pattern keys)
├── promotion-queue.json # bounded maintain queue for proven promotion candidates
├── projects/ # WARM tier (project-specific)
├── domains/ # WARM tier (domain-specific)
└── archive/ # COLD tier (inactive)
There are two different roots in this skill:
scripts/...references/...hooks/...learning/memory_tree/chunks.dblearning/index.md (SQLite-generated snapshot)memory/YYYY-MM-DD.md factual daily notes, when enabledlearning/projects/learning/domains/learning/archive/AGENTS.md, MEMORY.md, TOOLS.md, USER.md, SOUL.md, HEARTBEAT.md, IDENTITY.mdWhen SELF_IMPROVING_LEARNING_ROOT or --learning-root is set, the learning/* files above live in that shared learning root instead. Promotions still target the active workspace root, so a shared store can feed multiple projects without writing AGENTS/TOOLS files into the wrong workspace.
Never write learnings into the installed skill directory. Always target the workspace root for project memory and the optional learning root for shared lesson storage.
When this skill is installed in a persistent agent runtime, self-improvement must be enforced by the system rather than left to memory. The recommended architecture uses three enforcement loops:
search + log before every final reply after a non-trivial task. This catches lessons in real-time during active work.memory/, learning/, skills/, and the 7 root Markdown control-plane files aligned without broad rewrites.sessionTarget: "isolated" creates a fresh ephemeral session that does not pollute the main agent's context window or bust prompt-cache warmth.sessions_list / sessions_history can be used only when you have verified they work from isolated cron; otherwise they create false confidence.HEARTBEAT.md; keep that file minimal so heartbeat returns HEARTBEAT_OK quickly unless an urgent decision is needed.text
Cron Schedule (Asia/Shanghai)
────────────────────────────────────── ──────────────────────
Self-Improving Light Check 0 8-22/2 * * * (every 2h during waking hours)
Learning Audit (Heavy) 0 9,22 * * * (2x/day)
Daily Memory Digest 50 23 * * * (nightly factual memory)
Daily Workspace Steward 20 0 * * * (post-digest maintenance)
A quick in-between scan that reads recent conversation context and checks whether any user correction, non-obvious error, workaround, or tool/API quirk has been missed by the SQLite learning store. Preferred path: run SELF_IMPROVING_LIGHT_CONTEXT_COLLECTOR, read its exported Markdown/JSON, then dedupe with learnings.py search. Fallback path: use sessions_list / sessions_history only if verified from isolated cron. Tools: exec, read; optionally sessions_list, sessions_history for the fallback. Timeout: 120-180s.
Full audit: system-failure check, cron-failure scan, learning-audit.py --log, and learnings.py maintain --apply for lifecycle promotion/demotion. Tools: exec, read, cron. Timeout: 240s.
Run scripts/daily-memory.sh, gather or read the daily context, write memory/YYYY-MM-DD.md, then run the capture gate on the final note. Facts stay in memory/; compact reusable lessons go to SQLite. See references/daily-memory-digest.md. Timeout: 300s.
Run scripts/learning-export.sh, inspect learning/, skills/*/SKILL.md, and root agent markdown files for small safe consistency updates. It may fix verified stale facts or clear contradictions, but must not rewrite persona, weaken safety/privacy rules, delete files, or change cron jobs. Timeout: 300s.
The cron jobs described above are NOT created automatically when you install this skill. You must run the setup once to create them in your OpenClaw instance.
The production-grade cron job definitions are in scripts/setup-cron.json. The agent-facing setup guide is scripts/setup-cron-agent.md.
Quick setup (run from your OpenClaw main session):
Confirm: "I want to install the self-improving compound cron jobs. Use scripts/setup-cron.json as reference."
Your agent will:
cron add / cron update for each job, avoiding duplicatesVerify: cron list should show enabled jobs with nextRunAtMs set.
If you already have these jobs running, this step is a no-op.
Add the following rule to agent instructions (e.g. AGENTS.md):
Before every final reply after a non-trivial task: if the task involved a user correction, non-obvious failure, API/tool quirk, workaround, format mismatch, missing capability, or reusable convention, search existing SQLite learning first with
scripts/learnings.py --root <workspace> search "<keywords>" --limit 5. If no suitable entry exists, log the durable lesson before replying. Never rely on a mental note;learning/memory_tree/chunks.dbis the execution-learning source of truth.
Route watchdog, doctor, healthcheck, and cron failure signals into log-error with stable pattern keys and dedupe:
cron:<job-name>, doctor:<check-name>, watchdog:<component>, system:openclaw-audit-failurescripts/log-system-failures.sh as an OpenClaw CLI audit wrapper where available.HEARTBEAT_OK; do not echo full command output.SituationWhat to doUser corrects you or updates a factLog a correctionNon-obvious command / API / tool failureLog an errorUser asks for a missing capabilityLog a feature requestYou discover a reusable workaround or conventionLog a learningA pattern keeps recurringSearch related entries, link with See Also, and consider promotionA lesson is broadly applicable or repeatedPromote it into project memoryA resolved, general pattern could help other projectsExtract a new skill
If the workspace uses daily factual notes, run or schedule:
bash
bash scripts/daily-memory.sh --root /absolute/path/to/workspace
Then write memory/YYYY-MM-DD.md from the gathered context and run the capture gate on that note. Use references/daily-memory-digest.md for the quality bar. Do not copy the diary into SQLite; extract only reusable prevention rules.
Before reading or writing learning/, determine WORKSPACE_ROOT.
Good defaults:
OPENCLAW_WORKSPACE env var)If unsure, prefer the directory containing .git, AGENTS.md, CLAUDE.md, or the user's active project files.
learning/ if needed#Use the helper instead of creating files manually:
bash
python3 scripts/learnings.py --root /absolute/path/to/workspace init
This creates:
learning/memory_tree/chunks.db (SQLite database)learning/projects/learning/domains/learning/archive/The learning/index.md snapshot is generated on first write (log, promote, etc.).
Review first when:
Use the helper:
bash
python3 scripts/learnings.py --root /absolute/path/to/workspace status
python3 scripts/learnings.py --root /absolute/path/to/workspace search "pnpm" --limit 5
python3 scripts/learnings.py --root /absolute/path/to/workspace search "pnpm" --touch
# --root can also be placed after the subcommand
python3 scripts/learnings.py status --root /absolute/path/to/workspace --format json
Plain search is read-only. Use --touch only when the result was actually reused and should increment recurrence metadata.
Always search for related entries before creating a new one.
bash
python3 scripts/learnings.py --root /absolute/path/to/workspace search "keyword or pattern" --limit 10
If a similar entry already exists:
See AlsoPattern-Key for recurring issuesUse for user corrections and updated facts. Stored in SQLite with a human ID such as COR-YYYYMMDD-001.
bash
python3 scripts/learnings.py --root /absolute/path/to/workspace log-correction \
--summary "Used wrong format for Telegram" \
--correct "Use lists, not tables" \
--pattern chat:telegram-format
Use for corrections, knowledge gaps, best practices, and durable conventions. Stored in SQLite with a human ID such as LRN-YYYYMMDD-001.
bash
python3 scripts/learnings.py --root /absolute/path/to/workspace log-learning \
--summary "Project uses pnpm workspaces, not npm" \
--details "Attempted npm install. Lockfile and workspace config showed pnpm." \
--pattern pkg:pnpm-workspace
Use for non-obvious failures, exceptions, or tool/API issues worth remembering. Stored in SQLite with a human ID such as ERR-YYYYMMDD-001.
bash
python3 scripts/learnings.py --root /absolute/path/to/workspace log-error \
--summary "Docker build failed on Apple Silicon due to platform mismatch" \
--details "docker build -t myapp . on Apple Silicon" \
--pattern docker:platform
Use when the user wants a missing capability or a recurring friction point should become a feature. Stored in SQLite with a human ID such as FTR-YYYYMMDD-001.
bash
python3 scripts/learnings.py --root /absolute/path/to/workspace log-feature \
--summary "User needs report export to CSV" \
--details "Needed for sharing weekly reports with non-technical stakeholders" \
--pattern reports:csv-export
The old log subcommand is preserved for compatibility:
bash
python3 scripts/learnings.py --root /absolute/path/to/workspace log "Used wrong format" \
--type COR --pattern chat:telegram-format --correct "Use lists" --force
To inspect or share entries, export from SQLite:
bash
python3 scripts/learnings.py --root /absolute/path/to/workspace export
python3 scripts/learnings.py --root /absolute/path/to/workspace export --format json
Promote when the learning is broad, repeated, or something any future contributor should know.
Common targets:
CLAUDE.md — durable project facts and conventionsAGENTS.md — workflow rules and automation guidance.github/copilot-instructions.md — shared Copilot contextSOUL.md — behavioural principles in OpenClaw workspacesTOOLS.md — tool-specific gotchas in OpenClaw workspacesWrite promotions as short prevention rules, not long incident write-ups.
Example:
pnpm install in this repo; it is a pnpm workspace."When a learning is promoted, update the original entry's status to promoted or promoted_to_skill and record the destination.
maintain writes high-recurrence candidates to learning/promotion-queue.json. For fully automated deployments, maintain --apply --auto-promote promotes those candidates into the active workspace root while keeping the shared learning store separate when --learning-root is used.
Extract a new skill when the solution is:
Use the helper:
bash
bash scripts/extract-skill.sh my-skill-name /absolute/path/to/workspace
Entries carry metadata ( First-Seen, Last-Seen, Recurrence-Count, Status, Area) so the system can make deterministic lifecycle decisions without guessing.
TierLocationSize guidanceBehaviorHOTSQLite lifecycle admittedActive working setShown by status, search, and hooksWARMSQLite lifecycle buffered30+ days unusedRetained for context-specific searchCOLDSQLite lifecycle sealedarchived/promoted/resolvedRetained for explicit query/export
Use python3 scripts/learnings.py --root <workspace> maintain to review:
ConditionThresholdActionHOT -> WARM30 days unusedSet lifecycle to bufferedWARM -> COLD90 days unusedSet lifecycle to sealedFrequent reuseRecurrence-Count >= 3 from explicit reuse ( search --touch or edit)Flag for project-memory promotionCompaction/exportHuman review neededExport and manually summarize/promote without deleting the SQLite record
maintain defaults to --dry-run. Use --apply to execute lifecycle status moves. It never deletes content and does not auto-summarize.
When patterns contradict:
project > domain > globalConditionThresholdActionHOT -> WARM30 days unusedMark bufferedWARM -> COLD90 days unusedMark sealedFrequent reuse3 recorded uses within 7 daysPromote as a short prevention ruleTo AGENTS/SOUL/TOOLSRecurrence-Count >= 3 + spans 2+ tasks + within 30 daysPromote as short prevention ruleTo skillProven + broadly applicableExtract as skill
Use these only when needed:
references/entry-formats.md — full field schemas and manual templatesreferences/promotion-and-extraction.md — promotion rules and skill extraction criteriareferences/platform-setup.md — Claude Code, Codex, Copilot, and OpenClaw setup notesHook helpers are intentionally optional and workspace-root aware.
Available hook scripts:
hooks/activator.sh — lightweight reminder at prompt starthooks/error-detector.sh — lightweight error reminder after failed Bash-like commandsHook configuration examples live in references/platform-setup.md.
A mature use of this skill has a loop:
capture → dedupe → promote → extract → evaluate
That means:
evals/