Self-Improving Analytics Skill#

Log analytics-specific learnings, data issues, and feature requests to markdown files for continuous improvement. Captures data quality problems, metric drift, pipeline failures, misleading visualizations, metric definition mismatches, and data freshness breaches. Important learnings get promoted to data dictionaries, metric definitions, pipeline runbooks, dashboard standards, or data quality SLAs.

First-Use Initialisation#

Before logging anything, ensure the .learnings/ directory and files exist in the project or workspace root. If any are missing, create them:

bash

mkdir -p .learnings
[ -f .learnings/LEARNINGS.md ] || printf "# Analytics Learnings\n\nData quality patterns, metric drift insights, pipeline reliability findings, visualization best practices, and governance lessons.\n\n**Categories**: data_quality | metric_drift | pipeline_failure | visualization_mislead | definition_mismatch | freshness_issue\n**Areas**: ingestion | transformation | modeling | reporting | visualization | governance | data_catalog\n\n---\n" > .learnings/LEARNINGS.md
[ -f .learnings/DATA_ISSUES.md ] || printf "# Data Issues Log\n\nPipeline failures, data quality problems, metric anomalies, visualization errors, and schema drift.\n\n---\n" > .learnings/DATA_ISSUES.md
[ -f .learnings/FEATURE_REQUESTS.md ] || printf "# Feature Requests\n\nAnalytics tools, BI capabilities, data quality automation, and governance improvements.\n\n---\n" > .learnings/FEATURE_REQUESTS.md

Never overwrite existing files. This is a no-op if .learnings/ is already initialised.

Do not log connection strings, database credentials, API keys, or PII. Prefer short summaries or redacted excerpts over raw query results or full table dumps.

If you want automatic reminders, use the opt-in hook workflow described in Hook Integration.

Quick Reference#

SituationActionETL/ELT pipeline failsLog to .learnings/DATA_ISSUES.md with pipeline name and errorMetric value anomaly (spike/drop)Log to .learnings/DATA_ISSUES.md with statistical contextTwo teams define metric differentlyLog to .learnings/LEARNINGS.md with category definition_mismatchDashboard shows wrong or misleading dataLog to .learnings/LEARNINGS.md with category visualization_misleadData freshness SLA missedLog to .learnings/DATA_ISSUES.md with SLA threshold and actual delaySchema change breaks downstreamLog to .learnings/DATA_ISSUES.md with schema diff detailsNULL rate spike in key columnLog to .learnings/DATA_ISSUES.md with column and thresholdMetric silently drifts (calculation change)Log to .learnings/LEARNINGS.md with category metric_driftRecurring data quality patternLink with **See Also**, consider priority bumpBroadly applicable patternPromote to data dictionary, pipeline runbook, or dashboard standardReusable data quality checkPromote to data quality SLA or dbt test

OpenClaw Setup (Recommended)#

OpenClaw is the primary platform for this skill. It uses workspace-based prompt injection with automatic skill loading.

Installation#

Via ClawdHub (recommended):

bash

clawdhub install self-improving-analytics

Manual:

bash

git clone https://github.com/jose-compu/self-improving-analytics.git ~/.openclaw/skills/self-improving-analytics

Manual clone is optional. Check out a reviewed tag or commit before use. Do not install an unpinned default branch into a shared environment.

Workspace Structure#

OpenClaw injects these files into every session:

text

~/.openclaw/workspace/
├── AGENTS.md          # Multi-agent workflows, delegation patterns
├── SOUL.md            # Behavioral guidelines, personality, principles
├── TOOLS.md           # Tool capabilities, integration gotchas
├── MEMORY.md          # Long-term memory (main session only)
├── memory/            # Daily memory files
│   └── YYYY-MM-DD.md
└── .learnings/        # This skill's log files
    ├── LEARNINGS.md
    ├── DATA_ISSUES.md
    └── FEATURE_REQUESTS.md

Create Learning Files#

bash

mkdir -p ~/.openclaw/workspace/.learnings

Then create the log files (or copy from assets/):

• LEARNINGS.md — metric drift, definition mismatches, visualization issues, data quality patterns
• DATA_ISSUES.md — pipeline failures, freshness breaches, schema drift, metric anomalies
• FEATURE_REQUESTS.md — analytics tools, BI capabilities, automation requests

Promotion Targets#

When analytics learnings prove broadly applicable, promote them:

Learning TypePromote ToExampleMetric definitionsData dictionary"Active user = login within 7 days with feature interaction"Pipeline failure patternsPipeline runbooks"DST partition handling: always use UTC-based keys"Visualization standardsDashboard style guide"Absolute value charts must start Y-axis at zero"Data quality rulesData quality SLAs"NULL rate in PK columns must be <0.01%"Governance patternsAGENTS.md"New metrics require data dictionary entry before dashboard"Tool configurationTOOLS.md"dbt source freshness checks required on all external sources"

Optional: Enable Hook#

Opt-in and workspace-local. Do not copy this hook to ~/.openclaw/hooks/ (that path is user-global). Skip it unless you want reminders in this workspace:

bash

mkdir -p .openclaw/hooks
cp -r hooks/openclaw .openclaw/hooks/self-improving-analytics

See references/openclaw-integration.md for complete details.

---

Generic Setup (Other Agents)#

For Claude Code, Codex, Copilot, or other agents, create .learnings/ in the project or workspace root:

bash

mkdir -p .learnings

Create the files inline using the headers shown above.

Add reference to agent files#

Add to AGENTS.md, CLAUDE.md, or .github/copilot-instructions.md:

Self-Improving Analytics Workflow

When data issues or analytics patterns are discovered:

1. Log to .learnings/DATA_ISSUES.md, LEARNINGS.md, or FEATURE_REQUESTS.md
2. Review and promote broadly applicable learnings to:
   • Data dictionaries — canonical metric definitions
   • Pipeline runbooks — failure recovery procedures
   • Dashboard standards — visualization conventions
   • Data quality SLAs — monitoring thresholds and alerts

Logging Format#

Learning Entry [LRN-YYYYMMDD-XXX]#

Append to .learnings/LEARNINGS.md:

markdown

## [LRN-YYYYMMDD-XXX] category

**Logged**: ISO-8601 timestamp
**Priority**: low | medium | high | critical
**Status**: pending
**Area**: ingestion | transformation | modeling | reporting | visualization | governance | data_catalog

### Summary
One-line description of the analytics insight

### Details
Full context: what data pattern was found, why it is problematic,
what the correct approach is. Include root cause analysis.

### SQL Example

**Before (problematic):**
\`\`\`sql
-- problematic query, pipeline config, or metric definition
\`\`\`

**After (correct):**
\`\`\`sql
-- corrected query, config, or definition
\`\`\`

### Suggested Action
Specific data dictionary update, pipeline fix, dashboard change, or governance rule to adopt

### Metadata
- Source: etl_failure | freshness_breach | metric_anomaly | definition_conflict | dashboard_review | reconciliation_failure | schema_drift
- Pipeline: Airflow DAG name, dbt model, Fivetran connector (if applicable)
- Warehouse: snowflake | bigquery | redshift | postgres | databricks
- Related Tables: schema.table_name
- Tags: tag1, tag2
- See Also: LRN-20250110-001 (if related to existing entry)
- Pattern-Key: metric_drift.revenue_source | data_quality.null_spike (optional)
- Recurrence-Count: 1 (optional)
- First-Seen: 2025-01-15 (optional)
- Last-Seen: 2025-01-15 (optional)

---

Categories for learnings:

CategoryUse Whendata_qualityNULL spikes, duplicate records, invalid values, completeness issuesmetric_driftMetric calculation silently changed due to new data source, schema change, or logic updatepipeline_failureETL/ELT job failure, timeout, resource exhaustion, dependency issuevisualization_misleadChart axis, scale, aggregation, or color choice that misrepresents datadefinition_mismatchSame metric name with different definitions across teams or dashboardsfreshness_issueData arriving later than SLA, stale dashboards, partition delays

Data Issue Entry [DAT-YYYYMMDD-XXX]#

Append to .learnings/DATA_ISSUES.md:

markdown

## [DAT-YYYYMMDD-XXX] issue_type_or_name

**Logged**: ISO-8601 timestamp
**Priority**: high
**Status**: pending
**Area**: ingestion | transformation | modeling | reporting | visualization | governance | data_catalog

### Summary
Brief description of the data issue

### Error Output
\`\`\`
Actual error message, pipeline log, query error, or anomaly description (redacted/summarized)
\`\`\`

### Root Cause
What in the pipeline, data model, or source system caused this issue.
Include the problematic query or configuration.

### Fix
\`\`\`sql
-- corrected query, pipeline config, or data quality check
\`\`\`

### Prevention
How to avoid this issue in the future (data quality test, pipeline alert, schema validation, SLA monitor)

### Context
- Trigger: etl_failure | freshness_breach | metric_anomaly | null_spike | schema_drift | rendering_error
- Pipeline: Airflow DAG name, dbt model, Fivetran connector
- Warehouse: snowflake | bigquery | redshift | postgres | databricks
- Affected Tables: schema.table_name
- Downstream Impact: dashboards, reports, or teams affected

### Metadata
- Reproducible: yes | no | unknown
- Related Tables: schema.table_name
- See Also: DAT-20250110-001 (if recurring)

---

Feature Request Entry [FEAT-YYYYMMDD-XXX]#

Append to .learnings/FEATURE_REQUESTS.md:

markdown

## [FEAT-YYYYMMDD-XXX] capability_name

**Logged**: ISO-8601 timestamp
**Priority**: medium
**Status**: pending
**Area**: ingestion | transformation | modeling | reporting | visualization | governance | data_catalog

### Requested Capability
What analytics tool, automation, or capability is needed

### User Context
Why it's needed, what workflow it improves, what data problem it solves

### Complexity Estimate
simple | medium | complex

### Suggested Implementation
How this could be built: dbt macro, Airflow operator, data quality check, Looker feature, governance workflow

### Metadata
- Frequency: first_time | recurring
- Related Features: existing_tool_or_capability

---

ID Generation#

Format: TYPE-YYYYMMDD-XXX

• TYPE: LRN (learning), DAT (data issue), FEAT (feature request)
• YYYYMMDD: Current date
• XXX: Sequential number or random 3 chars (e.g., 001, A7B)

Examples: LRN-20250415-001, DAT-20250415-A3F, FEAT-20250415-002

Resolving Entries#

When an issue is fixed, update the entry:

1. Change **Status**: pending → **Status**: resolved
2. Add resolution block after Metadata:

markdown

### Resolution
- **Resolved**: 2025-01-16T09:00:00Z
- **Commit/PR**: abc123 or #42
- **Notes**: Added data quality test / updated pipeline runbook / fixed metric definition

Other status values:

• in_progress — Actively being investigated or fixed
• wont_fix — Decided not to address (add reason in Resolution notes)
• promoted — Elevated to data dictionary, pipeline runbook, or dashboard standard
• promoted_to_skill — Extracted as a reusable skill

Detection Triggers#

Automatically log when you encounter:

ETL/ELT Pipeline Failures (→ data issue with etl_failure trigger):

• Airflow DAG task failure or retry exhaustion
• dbt model build error or test failure
• Fivetran/Stitch sync failure or schema change warning
• Spark/Databricks job timeout or OOM
• CDC replication lag exceeding threshold

Data Freshness Breaches (→ data issue with freshness_breach trigger):

• Source data >24h stale (no new rows for today's partition)
• dbt source freshness check failure ( warn_after, error_after)
• Dashboard last-refresh timestamp exceeds SLA
• Downstream model depends on stale upstream

Metric Value Anomalies (→ data issue with metric_anomaly trigger):

• Metric value >3 standard deviations from 30-day rolling mean
• Day-over-day change exceeds historical maximum
• Metric drops to zero unexpectedly
• Revenue, user count, or conversion rate shows sudden discontinuity

NULL Rate Spikes (→ data issue with null_spike trigger):

• Key column NULL rate exceeds baseline by >1 percentage point
• Primary key contains NULLs (should never happen)
• Foreign key NULL rate spikes after source system change
• Required business fields (email, amount, status) contain unexpected NULLs

Schema Changes (→ data issue with schema_drift trigger):

• Column added, removed, or renamed in source system
• Data type changed (VARCHAR length, NUMERIC precision)
• Constraint added or removed (NOT NULL, UNIQUE, FK)
• Table renamed or moved to different schema

Conflicting Definitions (→ learning with definition_mismatch category):

• Same metric name in two dashboards with different values
• Team A and Team B disagree on metric logic
• Data dictionary entry contradicts actual implementation
• Looker explore definition differs from dbt model documentation

Visualization Issues (→ learning with visualization_mislead category):

• Y-axis not starting at zero for absolute value charts
• Dual Y-axis with mismatched scales
• Pie chart with too many slices (>6)
• Time series with inconsistent intervals
• Color coding that implies ranking where none exists

Priority Guidelines#

PriorityWhen to UseAnalytics ExamplescriticalWrong data in executive dashboard or regulatory reportRevenue under-reported to board, compliance data incorrect, PII exposure in dashboardhighPipeline down, metric definition conflict, SLA breachAirflow DAG failed for >4h, Marketing vs Product metric mismatch, daily report stalemediumData quality degradation, visualization improvementNULL rate trending up, dashboard axis misleading, catalog entry outdatedlowCatalog update, documentation, minor improvementColumn description missing, unused dashboard cleanup, tag standardization

Area Tags#

Use to filter learnings by analytics domain:

AreaScopeingestionData extraction, loading, CDC replication, API pulls, file importstransformationSQL transforms, dbt models, Spark jobs, data cleaning, deduplicationmodelingDimensional modeling, entity relationships, slowly changing dimensions, grainreportingScheduled reports, email digests, PDF generation, data exportsvisualizationDashboards, charts, Looker explores, Tableau workbooks, Metabase questionsgovernanceMetric definitions, data ownership, access control, PII classificationdata_catalogColumn descriptions, table documentation, lineage, tagging, search

Promoting to Permanent Analytics Standards#

When a learning is broadly applicable (not a one-off data fix), promote it to permanent standards.

When to Promote#

• Same metric definition conflict surfaces across 2+ teams
• Pipeline failure pattern recurs after different source system changes
• Visualization anti-pattern found in 3+ dashboards
• Data quality rule would have prevented multiple incidents
• Freshness monitoring gap caused repeated SLA breaches

Promotion Targets#

TargetWhat Belongs ThereData dictionaryCanonical metric definitions with owner, grain, and refresh cadencePipeline runbooksStep-by-step recovery for known failure patternsDashboard standardsVisualization conventions (axis, colors, aggregation rules)Data quality SLAsMonitoring thresholds and alert configurationsCLAUDE.mdProject-specific analytics conventions for AI agentsAGENTS.mdAutomated analytics workflows, data validation steps

How to Promote#

1. Distill the learning into a concise definition, rule, or procedure

2. Add to appropriate target (data dictionary entry, runbook step, SLA threshold)

3. Show a reviewed diff and apply only after explicit user approval

4. Update original entry:

   • Change **Status**: pending → **Status**: promoted
   • Add **Promoted**: data dictionary (or pipeline runbook, dashboard standard, data quality SLA)

Promotion Examples#

Learning → Data dictionary entry:

Marketing "active user" = 30-day login; Product = 7-day feature interaction → 420K vs 185K discrepancy.

Promoted as: active_users_30d (Marketing, login-based) and active_users_7d (Product, interaction-based) with governance note specifying which to use for board reports.

Learning → Pipeline runbook:

Pipeline fails every DST transition — partition key uses local time, hour 2 doesn't exist.

Promoted as: "DST Partition Recovery" runbook — rerun with UTC key, verify no duplicates, migrate all partitions to UTC.

Recurring Pattern Detection#

If logging something similar to an existing entry:

1. Search first: grep -r "keyword" .learnings/

2. Link entries: Add **See Also**: DAT-20250110-001 in Metadata

3. Bump priority if issue keeps recurring

4. Consider systemic fix: Recurring analytics issues often indicate:

   • Missing data quality test (→ add dbt test or Great Expectations suite)
   • Missing monitoring (→ add freshness check or anomaly detection)
   • Governance gap (→ add metric definition to data dictionary)
   • Pipeline design flaw (→ refactor ingestion or transformation logic)

Periodic Review#

Review .learnings/ at natural breakpoints:

When to Review#

• Before building a new dashboard or metric
• After a pipeline incident is resolved
• When the same data quality issue appears again
• Before quarterly business reviews
• During data model refactoring

Quick Status Check#

bash

# Count pending analytics issues
grep -h "Status\*\*: pending" .learnings/*.md | wc -l

# List pending high-priority data issues
grep -B5 "Priority\*\*: high" .learnings/DATA_ISSUES.md | grep "^## \["

# Find learnings for a specific area
grep -l "Area\*\*: governance" .learnings/*.md

# Find all definition mismatches
grep -B2 "definition_mismatch" .learnings/LEARNINGS.md | grep "^## \["

Review Actions#

• Resolve fixed data issues
• Promote recurring patterns to data dictionaries
• Link related entries across files
• Extract reusable data quality checks as skills

Simplify & Harden Feed#

Ingest recurring analytics patterns from simplify-and-harden into data quality rules or governance standards.

1. For each candidate, use pattern_key as the dedupe key.
2. Search .learnings/LEARNINGS.md for existing entry: grep -n "Pattern-Key: <key>" .learnings/LEARNINGS.md
3. If found: increment Recurrence-Count, update Last-Seen, add See Also links.
4. If not found: create new LRN-... entry with Source: simplify-and-harden.

Promotion threshold: Recurrence-Count >= 3, seen in 2+ pipelines/dashboards, within 30-day window. Targets: data dictionary entries, pipeline runbooks, dashboard standards, CLAUDE.md / AGENTS.md.

Hook Integration#

Enable automatic reminders through agent hooks. This is opt-in.

Hooks persist across sessions once installed. Keep them project-scoped. Do not install user-level or global hooks. Never use an empty matcher. PostToolUse inspects command output in-process; do not log raw output, secrets, or transcripts.

Quick Setup (Claude Code / Codex)#

Create .claude/settings.json in your project:

json

{
  "hooks": {
    "UserPromptSubmit": [{
      "matcher": "ETL|pipeline|metric|dashboard|freshness|schema|lineage|warehouse",
      "hooks": [{
        "type": "command",
        "command": "./skills/self-improving-analytics/scripts/activator.sh"
      }]
    }]
  }
}

This injects an analytics-focused learning evaluation reminder after matching prompts (~50-100 tokens overhead).

Advanced Setup (With Error Detection)#

json

{
  "hooks": {
    "UserPromptSubmit": [{
      "matcher": "ETL|pipeline|metric|dashboard|freshness|schema|lineage|warehouse",
      "hooks": [{
        "type": "command",
        "command": "./skills/self-improving-analytics/scripts/activator.sh"
      }]
    }]
  }
}

Leave PostToolUse disabled. Do not attach error-detector.sh to every Bash call.

Available Hook Scripts#

ScriptHook TypePurposescripts/activator.shUserPromptSubmitReminds to evaluate analytics learnings after tasksscripts/error-detector.shPostToolUse (Bash)Triggers on pipeline errors, query failures, data quality issues

See references/hooks-setup.md for detailed configuration and troubleshooting.

Automatic Skill Extraction#

Extracted skills are untrusted until a human reviews the generated SKILL.md. Do not keep or publish an extracted skill without explicit user approval.

When an analytics learning is valuable enough to become a reusable skill, extract it.

Skill Extraction Criteria#

CriterionDescriptionRecurringSame data issue in 2+ pipelines or warehousesVerifiedStatus is resolved with working fix and data quality testNon-obviousRequired actual investigation or cross-team coordinationBroadly applicableNot project-specific; useful across data stacksUser-flaggedUser says "save this as a skill" or similar

Extraction Workflow#

1. Identify candidate: Learning meets extraction criteria

2. Run helper (or create manually): bash

   ./skills/self-improving-analytics/scripts/extract-skill.sh skill-name --dry-run
   ./skills/self-improving-analytics/scripts/extract-skill.sh skill-name
   
   

3. Customize SKILL.md: Fill in template with analytics-specific content

4. Update learning: Set status to promoted_to_skill, add Skill-Path

5. Verify: Read skill in fresh session to ensure it's self-contained

Extraction Detection Triggers#

In conversation: "This pipeline keeps failing the same way", "Save this data quality check as a skill", "Every warehouse has this DST issue", "This metric definition problem happens everywhere".

In entries: Multiple See Also links, high priority + resolved, definition_mismatch or pipeline_failure with broad applicability, same Pattern-Key across projects.

Multi-Agent Support#

AgentActivationDetectionClaude CodeHooks (UserPromptSubmit, PostToolUse)Automatic via error-detector.shCodex CLIHooks (same pattern)Automatic via hook scriptsGitHub CopilotManual ( .github/copilot-instructions.md)Manual reviewOpenClawWorkspace injection + inter-agent messagingVia session tools

Best Practices#

1. Define metrics before building — agree on definitions in the data dictionary before creating dashboards
2. Validate at ingestion — catch data quality issues as early as possible in the pipeline
3. Test transformations with known data — use static test fixtures, not just production samples
4. Document lineage — every metric should trace back to source tables through explicit joins
5. Alert on anomalies, not just failures — a pipeline that succeeds but produces wrong data is worse than one that fails
6. Use UTC everywhere — local time in partitions, timestamps, or schedules causes DST and timezone bugs
7. Version metric definitions — treat metric logic changes like code changes with review and approval
8. Separate facts from interpretation — dashboards should present data; narrative belongs in annotations
9. Review before building in same area — check .learnings/ for past issues with the same tables or metrics
10. Reconcile against source of truth — compare warehouse aggregates against GL, CRM, or billing system totals

Gitignore Options#

Keep learnings local (per-analyst): add .learnings/ to .gitignore.

Track learnings in repo (team-wide): don't add to .gitignore — learnings become shared knowledge.

Hybrid (track templates, ignore entries): ignore .learnings/*.md, keep .learnings/.gitkeep.

Stackability Contract (Standalone + Multi-Skill)#

This skill is standalone-compatible and stackable with other self-improving skills.

Namespaced Logging (recommended for 2+ skills)#

• Namespace for this skill: .learnings/analytics/
• Keep current standalone behavior if you prefer flat files.
• Optional shared index for all skills: .learnings/INDEX.md

Required Metadata#

Every new entry must include:

markdown

**Skill**: analytics

Hook Arbitration (when 2+ skills are enabled)#

• Use one dispatcher hook as the single entrypoint.
• Dispatcher responsibilities: route by matcher, dedupe repeated events, and rate-limit reminders.
• Suggested defaults: dedupe key = event + matcher + file + 5m_window; max 1 reminder per skill every 5 minutes.

Narrow Matcher Scope (analytics)#

Only trigger this skill automatically for analytics signals such as:

• pipeline|etl|schema drift|metric mismatch|dashboard
• lineage|warehouse|bi|attribution|anomaly
• explicit analytics intent in user prompt

Cross-Skill Precedence#

When guidance conflicts, apply:

1. security
2. engineering
3. coding
4. ai
5. user-explicit domain skill
6. meta as tie-breaker

Ownership Rules#

• This skill writes only to .learnings/analytics/ in stackable mode.
• Do not read other skill folders, their SKILL.md files, or their log entries.
• Standalone mode writes to this project's .learnings/*.md log files only.
• Stackable mode writes only to the namespaced folder above and must not rewrite other skills' log entries.
• Promotion into AGENTS.md, SOUL.md, TOOLS.md, MEMORY.md, rules, hooks, or generated skills is not a logging write. Show a reviewed diff and apply only after explicit user approval.