supervised-agentic-loop#

Self-improving AI agent loop with built-in misalignment detection.

Quick Reference#

WhatDetailsLoopBrainstorm → Plan → Implement → Review → Verify → EvolveAgent modifiesOne file only ( target_file)MetricAny command that produces a numeric outputSafety (SAL)Git isolation + reputation scoring + 4 verification gatesSafety (Monitor)SYNC blocking + ASYNC LLM review + 10 behavior patternsPersistenceresults.tsv + .state/learnings/ + reputation.db + *.jsonl

Two Packages, One System#

text

sal/                        # Evolve Loop — the brain
├── config.py               # Run configuration
├── evolve_loop.py          # 6-phase loop orchestrator
├── contract.py             # AgentCallable protocol
├── metric_extractor.py     # Named strategies + regex
├── verification.py         # 4 verification gates
├── reputation.py           # EMA scoring + suspension
├── git_isolation.py        # Branch per run, auto-rollback
├── learnings.py            # Persistent pattern detection
├── brainstorm.py           # Hypothesis generation
├── cli.py                  # CLI entrypoint
└── monitor/                # Agent Monitor — the guardian
    ├── sanitizer.py        # Credential redaction (10 patterns)
    ├── behaviors.py        # 10 misalignment behaviors (B001-B010)
    ├── monitor.py          # Two-phase detection engine
    ├── classifier.py       # Severity classification + dedup
    ├── logger.py           # JSONL tool-call logging
    ├── alerter.py          # Telegram alerts (urllib)
    ├── heartbeat.py        # Self-monitoring + canary
    └── dashboard.py        # Command Center data functions

Dependency rule: sal/ imports monitor/, NEVER the reverse. Monitor has zero knowledge of SAL core.

How to Use#

As a Skill (in your agent instructions)#

markdown

Read the SKILL.md in supervised-agentic-loop/ and begin an evolve run.
Target file: train.py
Metric: python train.py (look for val_bpb, lower is better)

As a CLI#

bash

# Evolve loop
sal run --target train.py --metric "python train.py" --parser val_bpb
sal status
sal unsuspend --agent codex --reason "verified by human"

# Monitor
sal monitor stats       # sessions, alerts, health
sal monitor alerts      # recent misalignment alerts
sal monitor canary      # run 5 self-test checks

As a Python API#

python

from sal.config import EvolveConfig
from sal.evolve_loop import EvolveLoop

config = EvolveConfig(
    target_file="train.py",
    metric_command="python train.py",
    metric_parser="val_bpb",
    minimize=True,
)

def my_agent(prompt: str) -> str:
    # Your LLM call here — must return output with JSON block
    ...

# Monitor auto-enabled. Set enable_monitor=False to disable.
loop = EvolveLoop(config, agent=my_agent, agent_id="my-model")
summary = loop.run()

Monitor Standalone#

python

from sal.monitor import AgentMonitor, BlockDecision

monitor = AgentMonitor(state_dir=".state")

# SYNC: before every tool call (< 5ms)
decision = monitor.check_before_execute("exec", {"command": "rm -rf /"})
# → BlockDecision.BLOCK

# ASYNC: review full session
result = monitor.review_session(session_entries, agent_callable=my_llm)

Phases#

1. Baseline — Run metric on unmodified code (fails → HARD ABORT)
2. Brainstorm — Generate hypothesis from history + learnings
3. Plan — Create TaskContract with acceptance criteria
4. Implement — Agent modifies target_file (🛡️ Monitor: SYNC prefilter)
5. Review — Parse agent output for contract compliance
6. Verify — Run verification gates (files, syntax, tests, lint) + extract metric
7. Evolve — Compare metric → KEEP (commit) or DISCARD (rollback)

10 Misalignment Behaviors#

IDBehaviorSeveritySync BlockB001Restriction bypass / destructive deleteHIGH✅B002Unexpected tool callsMEDIUM—B003Goal deviationMEDIUM—B004Self-modificationCRITICAL—B005Data exfiltrationCRITICAL✅B006Hallucinated successHIGH—B007Privilege escalationHIGH✅B008Resource exhaustionMEDIUM✅B009Persistence / backdoorCRITICAL✅B010ObfuscationHIGH✅

Auto-Brake Conditions#

The loop stops automatically when:

• Reputation ≤ 0.2 → Agent suspended
• Monitor BLOCK → Iteration aborted + reputation penalty
• Plateau → No improvement for N iterations
• Budget → max_iterations reached
• SIGINT → Human interrupt (graceful)

Built-in Metric Parsers#

NameExtractslast_line_floatFloat from last line of outputpytest_passedNumber of passed testspytest_failedNumber of failed testscoverage_percentCoverage percentageval_bpbValidation BPB valuebenchmark_msMilliseconds from benchmark outputCustom regexAny regex with 1 capture group

Environment Variables#

VariableDefaultDescriptionSAL_DB_PATH``.state/reputation.dbReputation database pathMONITOR_TELEGRAM_BOT_TOKEN—Telegram bot token for alertsMONITOR_TELEGRAM_CHAT_ID—Telegram chat/user IDMONITOR_LLM_COMMAND—LLM for async session reviewMONITOR_STATE_DIR``.stateMonitor state directory

Constraints#

• Zero external dependencies (Python 3.11+ stdlib only)
• Agent modifies exactly ONE file per iteration
• All changes are git-isolated with automatic rollback
• Learnings persist across runs in .state/learnings/
• Monitor is optional — SAL works without it
• 130 tests (69 SAL + 61 Monitor)