EEAT OpenClaw Skill Audit#

AI Agent Skill Quality Assurance — This skill adapts the CORE-EEAT framework to evaluate OpenClaw Skills, ensuring they deliver meaningful utility while maintaining security and reliability.

Skill Overview#

OpenClaw Skills are modular capability extensions for AI agents, defined by SKILL.md files with YAML frontmatter and prompt instructions. This skill evaluates skill quality through 80 standardized criteria across 8 core dimensions, generating comprehensive audit reports including utility scores, security assessments, and actionable improvement recommendations.

Core Transformation:

• From: Install Skills blindly → Hope they work
• To: Systematic vetting → Data-driven skill selection

OpenClaw Skill Structure#

Every OpenClaw Skill consists of:

text

my-skill/
├── SKILL.md          # Core definition (YAML + Markdown instructions)
├── scripts/          # Optional executable scripts
│   └── main.py
└── references/       # Optional configuration and resources
    └── config.json

Key Components:

• YAML Frontmatter: Skill metadata (name, description, version, dependencies, gates)
• Prompt Instructions: How the AI should use this skill
• Scripts: Optional executable code for complex operations
• Gating Mechanism: Conditional activation (bins, env, os checks)

Applicable Scenarios#

Use this skill when users request:

Skill Selection#

• "Evaluate this skill before installing"
• "Compare two skills for the same task"
• "Which skill should I use for X?"

Security Vetting#

• "Is this skill safe to run?"
• "Scan for security vulnerabilities"
• "Check permission boundaries"

Skill Development#

• "Audit my skill for quality issues"
• "How to improve my skill's documentation?"
• "What security best practices am I missing?"

Skill Maintenance#

• "Review installed skills for quality"
• "Identify deprecated or risky skills"
• "Prioritize skill updates"

Core Capabilities#

This skill can:

1. Complete 80-Item Audit: Score each CORE-EEAT item adapted for OpenClaw Skills

2. Utility Scoring: Evaluate task completion efficiency and comparative value

3. Security Assessment: Three-level security evaluation (Pass/Caution/Risk)

4. Gating Validation: Check conditional activation requirements (bins, env, os)

5. Veto Item Detection: Flag critical security violations (command injection, data leakage)

6. Priority Ranking: Identify top 5 improvements by impact

7. Comparative Analysis: Compare skills for same use case

Skill Categories#

This skill supports 6 OpenClaw Skill types, each with different evaluation priorities:

Productivity Skills#

• Definition: Gmail, Calendar, Google Drive, Microsoft Office integration
• Focus: Task completion accuracy, API reliability, error handling
• Weights: C: 30% | R: 25% | Exp: 20% | Ept: 15% | O: 5% | E: 0% | A: 5% | T: 0%

Development Skills#

• Definition: Code generation, debugging, GitHub automation, CI/CD
• Focus: Code quality, correctness, testing, security best practices
• Weights: C: 25% | O: 20% | R: 20% | Ept: 20% | E: 5% | Exp: 5% | A: 5% | T: 0%

Research Skills#

• Definition: Web search, web fetch, document summarization, data analysis
• Focus: Information accuracy, source credibility, citation quality
• Weights: C: 25% | R: 25% | A: 20% | E: 15% | O: 10% | Exp: 0% | Ept: 0% | T: 5%

Automation Skills#

• Definition: Browser automation, file operations, shell commands, task scheduling
• Focus: Security, error handling, robustness, permissions
• Weights: T: 30% | C: 25% | R: 20% | O: 15% | Exp: 5% | Ept: 5% | E: 0% | A: 0%

Content Skills#

• Definition: Text generation, translation, image generation, audio processing
• Focus: Output quality, style consistency, creative value
• Weights: C: 30% | E: 25% | Exp: 20% | O: 15% | Ept: 5% | R: 5% | A: 0% | T: 0%

System Skills#

• Definition: System monitoring, resource management, network tools, debugging
• Focus: Performance, reliability, security, compatibility
• Weights: T: 25% | C: 20% | R: 20% | E: 15% | O: 10% | Ept: 5% | Exp: 5% | A: 0%

---

8 Progressive Quality Gates#

Gate 1: Metadata Validation (Pre-Installation)#

When: Before installing any skill

Duration: 2-5 minutes

Items:

• C01: YAML frontmatter present and valid
• C02: Skill name and description clear
• O01: Skill structure follows OpenClaw convention
• T04: No suspicious dependencies or permissions

Deliverable: Metadata Validation Report

Failure: Do not install. Contact skill author or fix manually.

---

Gate 2: Gating Mechanism Check#

When: After metadata validation, before activation

Duration: 1-2 minutes

Items:

• O02: Required tools exist (bins check)
• O03: Required environment variables set (env check)
• O04: OS compatibility verified (os check)
• T07: No conflicting permissions

Deliverable: Gating Compatibility Report

Failure: Skill will not activate. Fix environment or choose alternative.

---

Gate 3: Security Pre-Check#

When: Before first execution

Duration: 3-5 minutes

Items:

• T01: No command injection vulnerabilities
• T02: No data leakage risks
• T03: Input validation present
• T04: Permissions are minimal principle

Deliverable: Security Pre-Check Report

Failure: Do not execute. Review code or choose alternative.

---

Gate 4: Prompt Quality Review#

When: During skill development or installation

Duration: 5-10 minutes

Items:

• C03: Instructions are clear and actionable
• C04: Tool usage patterns are explicit
• Ept03: Professional terminology used
• Ept05: Follows prompt engineering best practices

Deliverable: Prompt Quality Report

Failure: Skill may misbehave. Refine SKILL.md instructions.

---

Gate 5: Script Security Audit#

When: For skills with executable scripts

Duration: 10-20 minutes

Items:

• T05: No hardcoded secrets or API keys
• T06: No eval/exec of user input
• T07: Proper error handling and logging
• T08: Resource cleanup implemented

Deliverable: Script Security Audit

Failure: Security risk. Audit scripts or avoid skill.

---

Gate 6: Utility Testing#

When: During skill evaluation

Duration: 15-30 minutes

Items:

• C05: Completes intended tasks successfully
• C06: Output quality meets expectations
• Exp01: Usage examples provided and working
• Exp02: Performance characteristics documented

Deliverable: Utility Test Report

Failure: Skill doesn't deliver value. Consider alternatives.

---

Gate 7: Comparative Analysis#

When: Selecting between multiple skills for same task

Duration: 10-15 minutes

Items:

• E01: Offers unique value vs alternatives
• E02: Better performance or efficiency
• A01: Active maintenance and updates
• A02: Community adoption and reviews

Deliverable: Comparative Analysis Report

Failure: Alternative skill may be better. Choose based on scores.

---

Gate 8: Full 80-Item Audit#

When: Critical skills, major updates, security reviews

Duration: 30-60 minutes

Items: All 80 items across all 8 dimensions

Deliverable: Comprehensive EEAT Audit Report

Failure: Address critical issues before production use.

---

Usage Examples#

Example 1: Pre-Installation Security Check#

User Request:

text

Check if this skill is safe to install:
clawhub install agent-browser

Response:

markdown

## EEAT Skill Security Audit: agent-browser

### Gate 1: Metadata Validation ✅
- [x] YAML frontmatter valid
- [x] Description clear: "Browser automation for web scraping"
- [x] Version: 1.2.3
- [x] Dependencies: playwright (verified safe)

### Gate 2: Gating Check ⚠️
- [x] Bins: playwright installed
- [x] Env: PLAYWRIGHT_BROWSERS_PATH set
- [ ] OS: macOS/Windows only (Linux limited support)

### Gate 3: Security Pre-Check ✅
- [x] T01: No command injection detected
- [x] T02: No data leakage risks
- [x] T03: Input validation on URLs
- [x] T04: Minimal permissions (browser control only)

### Security Score: 92/100
**Security Status**: CAUTION ⚠️
- Linux support limited
- Requires browser installation (system-level change)

### Recommendation
✅ **Install with Caution**
- Safe for macOS/Windows users
- Linux users should check compatibility
- Review permissions before first use

---

Example 2: Compare Two Skills#

User Request:

text

Compare tavily-search vs web-search for web search

Response:

markdown

## Skill Comparative Analysis: Web Search

### Skill A: tavily-search

**Utility Score**: 85/100
- C05: High accuracy search results ✅
- C06: Clean, structured output ✅
- Exp01: Excellent usage examples ✅
- E01: AI-optimized for agents 🎯

**Security Score**: 95/100
- T01-T04: All checks pass ✅
- T07: No external dependencies ✅

**Overall**: 88/100 (Very Good)

### Skill B: web-search (bundled)

**Utility Score**: 70/100
- C05: Basic search functionality ✅
- C06: Output quality varies ⚠️
- Exp01: Limited examples ⚠️
- E01: Generic implementation ⚠️

**Security Score**: 90/100
- T01-T04: Safe but basic ✅
- T07: Uses external APIs ⚠️

**Overall**: 75/100 (Good)

### Recommendation
✅ **Choose tavily-search**
- +18 points higher utility
- AI-optimized for agent workflows
- Better structured output for downstream processing
- Higher security score

**Trade-off**: Requires API key setup vs. no setup for web-search

---

Example 3: Full Skill Audit#

User Request:

text

Audit my custom skill: email-automation
[Paste SKILL.md content]

Response:

markdown

## EEAT Skill Audit: email-automation

### Overview
- **Skill Type**: Productivity
- **Total Score**: 62/100 (Fair)
- **Utility Score**: 75/100 (Good)
- **Security Score**: 45/100 (CAUTION ⚠️)
- **Veto Item**: ⚠️ T02 Triggered (Data Leakage Risk)

### Dimension Scores

| Dimension | Score | Rating | Weight | Weighted Score |
|-----------|-------|--------|--------|----------------|
| C — Correctness | 70/100 | Good | 30% | 21 |
| O — Organization | 65/100 | Fair | 5% | 3.25 |
| R — Referenceability | 50/100 | Fair | 25% | 12.5 |
| Exp — Experience | 80/100 | Good | 20% | 16 |
| Ept — Expertise | 55/100 | Fair | 15% | 8.25 |
| E — Exclusivity | 40/100 | Poor | 0% | 0 |
| A — Authority | 60/100 | Fair | 5% | 3 |
| T — Trust | 45/100 | Poor | 0% | 0 |
| **Weighted Total** | | | | **64** |

### Critical Issues (Veto Items)

⚠️ **T02: Data Leakage Risk**
**Issue**: Skill stores API credentials in plain text in SKILL.md
```yaml
# SKILL.md
credentials:
  smtp_password: "mypassword123"  # ⚠️ SECURITY RISK

Action: Move credentials to environment variables

yaml

credentials:
  smtp_password: "${SMTP_PASSWORD}"  # ✅ SECURE

Top 5 Priority Improvements#

1. T02 Data Leakage — Remove hardcoded credentials
   • Current: Fail | Potential Gain: 8 weighted points
   • Action: Use environment variables for all secrets
2. R02 Coverage — Add error handling examples
   • Current: Fail | Potential Gain: 6.25 weighted points
   • Action: Document error scenarios and recovery
3. Ept01 Documentation — Improve prompt instructions
   • Current: Partial | Potential Gain: 4.5 weighted points
   • Action: Add step-by-step usage examples
4. R03 Source Authority — Verify email library security
   • Current: Partial | Potential Gain: 3.75 weighted points
   • Action: Audit nodemailer dependency for vulnerabilities
5. O01 Structure — Add scripts/ directory for complex logic
   • Current: Partial | Potential Gain: 2.5 weighted points
   • Action: Move complex operations to Python scripts

Action Plan#

Quick Wins (Fix immediately)

• Move all credentials to environment variables
• Add error handling documentation

Medium Investment (This week)

• Add comprehensive usage examples
• Implement proper logging in scripts

Strategic (Next sprint)

• Add test suite with edge cases
• Implement retry logic for failed sends
• Add HTML email support

Recommendation#

⚠️ Do Not Install Until Fixed

• Security risk (T02 veto) must be addressed
• After fixes, expected score: 78/100 (Good)

text

---

## Reference Documents

- `references/openclaw-skill-benchmark.md` — Complete 80-item benchmark adapted for OpenClaw Skills
- `references/skill-security-checklist.md` — Security-specific evaluation criteria
- `references/utility-testing-guide.md` — How to test skill utility and comparative value
- `workflow-optimization-analysis.md` — Adaptation strategy from code to skills

---

## Key Differences: Code vs. Skill Audit

| Aspect | Code Audit | Skill Audit |
|--------|-----------|-------------|
| **Primary Focus** | Code correctness, maintainability | Utility, security, reliability |
| **Security Emphasis** | SQL injection, XSS | Command injection, data leakage, permissions |
| **Evaluation Method** | Static analysis + testing | Comparative utility + security probes |
| **Output Format** | Code quality report | Utility score + security status label |
| **Key Metrics** | Test coverage, complexity | Task completion, risk level |
| **Veto Items** | Security bugs, logic errors | Security vulnerabilities, data risks |
| **Automation Level** | High (linters, type checkers) | Medium (requires manual security review) |
| **Comparative Analysis** | Code vs. requirements | Skill vs. baseline/skills |

---

## Success Points

1. **Security-First Approach** — OpenClaw Skills have system-level access; security is non-negotiable

2. **Comparative Utility** — Evaluate skills relative to baseline, not in isolation

3. **Gating Validation** — Ensure skills only activate when dependencies are met

4. **Prompt Quality** — SKILL.md instructions determine skill behavior; quality matters

5. **Minimal Permissions** — Skills should only request necessary access

6. **Active Maintenance** — Prioritize skills with recent updates and community support

7. **Real-World Testing** — Test with actual use cases, not synthetic scenarios

---

## Optimization Recommendations

Based on OpenClaw's architecture and community best practices:

### 1. Add Skill Registry Integration
- Integrate with ClawHub API for real-time skill metadata
- Auto-fetch version history, update frequency, download counts
- Community signals: stars, issues, last commit date

### 2. Implement Automated Security Scanning
- Integrate with `clawsec` (ClawHub security scanner)
- Auto-scan scripts/ directory for vulnerabilities
- Check for hardcoded secrets, eval/exec patterns

### 3. Add Utility Benchmarking
- Run comparative tests: baseline vs. with-skill
- Measure task completion time, token efficiency, success rate
- Generate utility scores similar to SkillTester framework

### 4. Create Skill Dependency Graph
- Map skill dependencies (some skills require others)
- Detect circular dependencies
- Recommend optimal skill combinations

### 5. Implement Skill Conflict Detection
- Detect skills with conflicting tool usage
- Identify resource contention (browser, file locks)
- Suggest skill compatibility matrix

### 6. Add Performance Profiling
- Track skill execution time over sessions
- Monitor API usage and costs
- Identify bottlenecks in skill chains

### 7. Create Skill Reputation System
- Track skill reliability across users
- Aggregate success/failure rates
- Community-rated skill quality scores

### 8. Implement A/B Testing Framework
- Compare two skills for same task
- Measure which completes faster/better
- Data-driven skill selection

### 9. Add Skill Update Notifications
- Monitor skill updates in ClawHub
- Alert on breaking changes
- Suggest upgrade timing

### 10. Create Skill Usage Analytics
- Track which skills are used most frequently
- Identify skill chains and workflows
- Optimize skill loading order

---

## Notes

- This skill adapts EEAT from content/code evaluation to AI agent skill vetting
- Security is elevated to critical importance due to system-level access
- Utility is evaluated comparatively (skill vs. baseline), not absolutely
- Gating mechanisms ensure skills only activate when dependencies are met
- Community signals (downloads, stars, issues) inform Authority dimension
- Prompt quality in SKILL.md directly impacts skill behavior
- Skills with executable scripts require deeper security audit
- This framework is designed for OpenClaw's modular skill architecture