ClawCheck#

Two-phase audit: a fast deterministic scan catches structural issues, then you (the agent) do a deep quality evaluation on the flagged areas.

When to Use#

• After initial setup or major config changes
• Before publishing skills to ClawHub (quality gate)
• Periodic health check (weekly cron or manual)
• When something feels off but openclaw doctor says "ok"
• After installing new skills or updating OpenClaw

What This Checks vs Built-in#

This skillopenclaw doctor (built-in)Secrets exposure + token hygieneConfig JSON schema validationCron ops health + prompt quality reviewPlugin/skill eligibilityConfig optimization + value assessmentChannel connectivitySkill structural + content quality auditState migrations, browser detection

How It Works: Two Phases#

Phase 1: Deterministic Scan (fast, free)#

Run the script to get a structural baseline:

bash

python3 {baseDir}/scripts/audit.py

Individual modules:

bash

python3 {baseDir}/scripts/audit.py --security
python3 {baseDir}/scripts/audit.py --cron
python3 {baseDir}/scripts/audit.py --config
python3 {baseDir}/scripts/audit.py --skills

This produces JSON with scores, findings, and the bottom/top skill lists. Use this as your triage map for Phase 2.

Phase 2: Deep Quality Audit (you, the agent)#

After running the script, perform these evaluations. Budget your depth based on what the user asked for ("quick check" = Phase 1 only, "full audit" or "quality review" = both phases).

2a. Config Quality Review

Read ~/.openclaw/openclaw.json and evaluate:

• Heartbeat prompt: Read agents.defaults.heartbeat.prompt. Is it specific enough to catch real issues? Does it avoid heavy operations? A good heartbeat prompt is < 200 words, checks 2-3 things, and has clear escalation criteria.
• Model choices: Is the primary model appropriate for the workload? Are fallbacks a meaningful step-down (not the same tier)? Is the subagent model cheaper than primary?
• Compaction thresholds: Are reserveTokens and keepRecentTokens reasonable for the context window size? Rule of thumb: reserve should be 15-20% of contextTokens.
• Session maintenance: Are pruneAfter, maxEntries, rotateBytes set to values that match the usage pattern? Heavy cron usage needs more aggressive pruning.
• Cron maxConcurrentRuns: Is it high enough for the number of frequent jobs? Count jobs with */ in their schedule expression.

Score each aspect 1-5. Report specific improvements.

2b. Cron Prompt Quality Review

Read ~/.openclaw/cron/jobs.json. Select the 5 most important enabled jobs using this heuristic:

1. Any job in error state (from Phase 1 findings)
2. Jobs with highest frequency x timeoutSeconds (most resource-consuming)
3. Jobs running on expensive models (opus/primary)
4. If still under 5, pick by business impact (backups, monitoring, user-facing)

For each selected job evaluate:

• Prompt clarity: Specific enough to execute without guessing? Clear steps, expected output format, error handling?
• Safety: Has guardrails? ("NEVER run git push", "read-only", "do not edit files directly")
• Efficiency: Token-efficient? Flag prompts > 1500 chars that run on expensive models. Could the prompt reference a skill file instead of inlining instructions?
• Output value: Produces actionable output or just noise?
• Timeout: payload.timeoutSeconds set and reasonable for scope?

Score each job 1-5 on: purpose, prompt quality, safety, efficiency. Flag jobs scoring below 3.

Cross-reference: Check if any cron prompts reference skills that scored below 70 in Phase 1. A cron job is only as reliable as the skills it depends on.

2c. Skill Content Quality Review

From the Phase 1 results, pick:

• The 3 lowest-scoring skills (from bottom_5)
• Any skills the user specifically asks about
• Skills used by failing cron jobs (cross-reference cron findings)

For each selected skill, read its full SKILL.md and evaluate:

• Accuracy (2x weight): Would following these instructions produce correct behavior? Are API references current? Are file paths real?
• Completeness (1.5x): Are all use cases covered? Edge cases? What happens when dependencies are missing?
• Clarity (1x): Can an agent follow this without ambiguity? No hedging, clear steps, good examples?
• Efficiency (1x): Is the SKILL.md bloated? Could it be shorter without losing information? Does it suggest efficient patterns (batching, caching)?
• Voice alignment (1x, content-producing skills only): Does the output match the brand/user's tone?

Scoring formula depends on skill type:

• Content/marketing skills (has voice component): (accuracy*2 + completeness*1.5 + clarity + efficiency + voice) / 6.5
• Utility/tool skills (no voice): (accuracy*2 + completeness*1.5 + clarity + efficiency) / 5.5

For skills scoring below 4.0, write specific improvement recommendations with concrete examples.

2d. Security Assessment

Phase 1 now scans workspace files for common secret patterns (sk-, ghp_, AIzaSy, Bearer tokens, hex private keys, etc.). In Phase 2, go deeper:

• Review any secrets the script found in workspace files. Are they real credentials or false positives (e.g., example/placeholder values)?
• Check if any skill scripts/ contain hardcoded credentials or API URLs with embedded tokens
• Check if .env files exist inside skill directories
• Look for credentials in cron job prompts (some prompts inline API keys instead of referencing env vars)
• Check if any workspace knowledge files contain customer data, passwords, or access tokens

Output Format#

Phase 1 (script output)#

json

{
  "score": 82,
  "score_type": "structural_hygiene",
  "status": "healthy",
  "sections": {
    "security": {"score": 65, "finding_count": 3},
    "cron": {"score": 95, "finding_count": 1},
    "config": {"score": 88, "finding_count": 2},
    "skills": {"score": 80, "finding_count": 1}
  },
  "findings": [...]
}

Phase 2 (your evaluation)#

Present as a readable report to the user:

text

## ClawCheck Report

### Structural Baseline (Phase 1)
Overall: 82/100 (healthy)
Security: 65 | Cron: 95 | Config: 88 | Skills: 80

### Deep Quality Findings (Phase 2)

**Config:**
- Heartbeat prompt: 4/5 (clear but could add Telegram alert on critical)
- Model choices: 5/5 (opus primary, sonnet fallback, sonnet subagent)
- Compaction: 4/5 (reserveTokens=150k for 800k context = 19%, good)

**Cron (top concerns):**
- "Morning Brief" (3/5): prompt is 400 words but lacks output format spec
- "Bleeding Edge Scanner" (2/5): no safety guardrails, no error handling

**Skills (bottom 3):**
- marketing-automation: BROKEN (no SKILL.md)
- apple-notes (62/100 structural): [content evaluation]
- blucli (62/100 structural): [content evaluation]

### Recommended Actions (priority order)
1. [most impactful fix]
2. [next fix]
3. [next fix]

Scoring Weights (Phase 1 script)#

Security 30%, cron 25%, config 20%, skills 25%.

Skill structure formula: (structure*2 + completeness*1.5 + clarity + efficiency) / 5.5 * 20

Remediation#

For detailed fix patterns with real config examples, see {baseDir}/references/remediation.md.

Quick fixes for common findings:

Inline secrets#

json

"GAMMA_API_KEY": {"source": "exec", "provider": "op-gamma", "id": "value"}

Plaintext bot token#

json

"botToken": {"source": "exec", "provider": "op-telegram", "id": "value"}

Missing heartbeat#

json

"heartbeat": {"every": "1h", "model": "sonnet", "prompt": "HEARTBEAT: Quick check..."}

Missing timezone on cron#

json

"schedule": {"kind": "cron", "expr": "0 9 * * *", "tz": "Europe/Madrid"}

Error Handling#

• If OpenClaw dir not found: script exits with error JSON and exit code 1.
• If openclaw.json is missing or invalid: script exits with error JSON.
• If individual module fails: caught and reported as warning, other modules still run.
• If bundled skills dir not accessible: skipped silently.
• Phase 2 failures: if you can't read a file, note it and move on. Don't stop the whole audit.

Non-Goals#

• No direct edits to config or skills (report only, user decides)
• No network calls (everything is local file inspection)
• No overlap with openclaw doctor schema validation or channel connectivity checks