n8n Pilot ⚙️#

Design, build, deploy, test, and secure advanced n8n workflows. Not an API wrapper — a workflow architect that guides you from intent to production.

When to Use#

Use when the task involves n8n — designing workflows, building automations, deploying instances, diagnosing failures, optimizing performance, or securing webhooks/credentials.

Companion Skills#

• clawhub install n8n — API client for CRUD operations (list/create/activate workflows)
• clawhub install docker-pilot — Container lifecycle management for deploying n8n

---

Safety Architecture ⚠️#

Dangerous Pattern Detection#

Before deploying ANY workflow, scan for these patterns:

PatternRiskDetectionLoop without max iteration🔴 Infinite executionSplit In Batches without batchSize, or Code node with while(true)Delete without filter🔴 Data destructionDatabase delete/HTTP DELETE without WHERE/filter parametersWebhook without auth🟡 Anyone can triggerWebhook node with authentication=noneHTTP to private IPs🟡 SSRF riskHTTP Request to 10.x, 172.16-31.x, 192.168.x, localhostEmail/Send without limit🟡 Mass spamLoop + email/Send node without max capNo error handling🟡 Silent failuresWorkflow with no Error Trigger or error branchesHardcoded secrets🔴 Credential exposureCode node containing API keys, passwords, tokens in plaintext

Rule: Workflows with 🔴 patterns MUST be fixed before deployment. 🟡 patterns require user acknowledgment.

Confirmation Gates#

• New workflow deployment: Show logic map → confirm → deploy inactive → test → activate
• Workflow modification: Show diff (what changed) → confirm → deploy
• Destructive operations (delete workflow, prune executions): Full audit + explicit confirmation

Kill Switch#

If a workflow goes haywire:

bash

# Deactivate all workflows immediately
python3 scripts/n8n_api.py list-workflows --active true | \
  python3 -c "import sys,json; [print(w['id']) for w in json.load(sys.stdin)['data']]" | \
  xargs -I{} python3 scripts/n8n_api.py deactivate --id {}

---

Workflow Design: Intent → Logic Map → JSON#

Step 1: Understand Intent#

Before building, articulate the workflow in plain language:

text

"I want to be notified on Telegram when an important email arrives from my boss."

Step 2: Create Logic Map#

Translate intent into a logic map (show to user for confirmation):

text

📧 Email Trigger (new email)
  → 🧠 IF node (sender contains "boss@company.com")
    → ✅ Yes: Summarize email content
      → 📱 Telegram notification (summary)
    → ❌ No: Skip

Step 3: Generate Workflow JSON#

Convert the logic map into n8n workflow JSON structure:

json

{
  "name": "Boss Email → Telegram Alert",
  "nodes": [...],
  "connections": {...},
  "settings": {
    "executionOrder": "v1",
    "timezone": "Europe/Berlin"
  }
}

Key JSON fields:

• typeVersion — must match the node version installed in your n8n instance
• position — [x, y] coordinates for UI rendering (required by API)
• credentials — embedded as {credential_type: {id, name}}, not just an ID
• parameters — node-specific config, use = prefix for expressions
• settings.executionOrder — must be "v1" for modern n8n
• pinData — inject mock data per-node for testing

---

Flow Logic Patterns#

Branching (IF / Switch)#

text

Trigger → IF (condition)
  → True branch: [process A]
  → False branch: [process B or Stop]

IF Node config:

json

{
  "type": "n8n-nodes-base.if",
  "parameters": {
    "conditions": {
      "boolean": [{
        "value1": "={{ $json.sender }}",
        "operation": "contains",
        "value2": "boss@company.com"
      }]
    }
  }
}

Switch Node (multi-way branching):

json

{
  "type": "n8n-nodes-base.switch",
  "parameters": {
    "rules": [
      { "output": 0, "conditions": {...} },
      { "output": 1, "conditions": {...} }
    ]
  }
}

Merging (Merge Node)#

text

Branch A → Merge → Continue
Branch B ↗

Modes:

• append — combine all items from all inputs
• mergeByPosition — zip items by index
• mergeByKey — join on a shared field (like SQL JOIN)

Looping (Split In Batches / Loop Over Items)#

text

Large Dataset → Split In Batches (size=100)
  → Process batch
  → Loop back
→ Continue when done

⚠️ Always set batchSize — prevents memory exhaustion on large datasets.

Sub-Workflows (Execute Workflow Node)#

text

Main Workflow → Execute Workflow (sub-workflow ID)
  → Use sub-result in main flow

Use cases: Reusable logic (e.g., "send notification" sub-workflow called by 5 different main workflows), breaking complex flows into manageable pieces.

Important: Sub-workflow must have a trigger that accepts calls from parent ( Workflow Trigger node).

Error Handling (Error Trigger + Error Branches)#

Pattern 1: Error Trigger Node

text

[Separate workflow]
Error Trigger → Notify (Telegram/Email) with error details

Pattern 2: Per-node error branch

text

HTTP Request → [on error] → Log error + fallback action
             → [on success] → Continue normal flow

Retry with Exponential Backoff#

n8n has no built-in retry with backoff. Implement in Code node:

javascript

// In a Code node, wrapping an API call
const maxRetries = 3;
const baseDelay = 1000; // 1 second

for (let attempt = 1; attempt <= maxRetries; attempt++) {
  try {
    const response = await this.helpers.httpRequest({
      url: 'https://api.example.com/data',
      method: 'GET',
    });
    return [{ json: response }];
  } catch (error) {
    if (attempt === maxRetries) throw error;
    const delay = baseDelay * Math.pow(2, attempt - 1);
    await new Promise(resolve => setTimeout(resolve, delay));
  }
}

---

Core Node Catalog#

Trigger Nodes (Start a Workflow)#

NodeUse WhenKey ConfigManual TriggerTesting, one-off runsNo config — click "Execute"WebhookExternal systems calling n8nPath, method (GET/POST), authenticationSchedule TriggerRecurring (cron) tasksCron expression or intervalEmail Trigger (IMAP)New email arrivesIMAP credentials, folder, filterPolling TriggerCheck API periodicallyURL, interval, pagination

Flow Control Nodes#

NodeUse WhenKey ConfigIFBinary branchingConditions (boolean/string/number)SwitchMulti-way branchingRules with output indicesMergeCombining branchesMode (append/mergeByPosition/mergeByKey)Split In BatchesProcess large datasets in chunksBatch size (always set this!)Loop Over ItemsProcess items one at a timeCan use Split In Batches insteadWaitDelay executionTime (seconds) or specific timeNo OperationPlaceholder / passthroughDoes nothing — useful for routing

Action Nodes#

NodeUse WhenKey ConfigHTTP RequestAny REST/GraphQL APIURL, method, headers, auth, bodyCode (JavaScript)Custom logic, transformationInput items, access via $inputSetAdd/modify fieldsField assignments, expressionsDate & TimeTimezone conversion, formattingFormat string, timezoneCryptoHash, encrypt, signAlgorithm, keySpreadsheet FileRead/write Excel/CSVFile format, options

App Nodes (Common Integrations)#

NodeForAuth TypeGmailSend/read emailsOAuth2Google SheetsRead/write spreadsheetsOAuth2Google CalendarEvent managementOAuth2TelegramSend messages, botsBot tokenSlackMessages, channelsOAuth2 or Bot tokenPostgreSQLDatabase queriesConnection stringMySQLDatabase queriesConnection stringMongoDBDocument queriesConnection stringRedisKey-value operationsConnection stringGitHubRepos, issues, PRsPersonal access token or OAuth2DiscordMessages, webhooksBot token

---

Self-Hosting Guide#

Development (Single Container)#

yaml

# docker-compose.yml
services:
  n8n:
    image: docker.n8n.io/n8nio/n8n
    container_name: n8n
    restart: unless-stopped
    ports:
      - "5678:5678"
    environment:
      - N8N_ENCRYPTION_KEY=<generate-a-random-32-char-key>
      - WEBHOOK_URL=http://localhost:5678
      - TZ=Europe/Berlin
    volumes:
      - n8n_data:/home/node/.n8n
      - /etc/localtime:/etc/localtime:ro

volumes:
  n8n_data:

Production (Queue Mode — PostgreSQL + Redis)#

yaml

# docker-compose.yml — production
services:
  postgres:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER: n8n
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_DB: n8n
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U n8n"]
      interval: 30s
      timeout: 5s
      retries: 3

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    command: redis-server --appendonly yes --maxmemory 256mb --maxmemory-policy allkeys-lru
    volumes:
      - redis_data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 30s
      timeout: 5s
      retries: 3

  n8n-main:
    image: docker.n8n.io/n8nio/n8n
    container_name: n8n
    restart: unless-stopped
    ports:
      - "5678:5678"
    environment:
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_DATABASE=n8n
      - DB_POSTGRESDB_USER=n8n
      - DB_POSTGRESDB_PASSWORD=${DB_PASSWORD}
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - N8N_ENCRYPTION_KEY=${ENCRYPTION_KEY}
      - WEBHOOK_URL=https://n8n.example.com
      - TZ=Europe/Berlin
      - EXECUTIONS_DATA_PRUNE=true
      - EXECUTIONS_DATA_MAX_AGE=336
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    volumes:
      - n8n_data:/home/node/.n8n

  n8n-worker:
    image: docker.n8n.io/n8nio/n8n
    restart: unless-stopped
    command: worker --concurrency=5
    environment:
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_DATABASE=n8n
      - DB_POSTGRESDB_USER=n8n
      - DB_POSTGRESDB_PASSWORD=${DB_PASSWORD}
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - N8N_ENCRYPTION_KEY=${ENCRYPTION_KEY}
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy

volumes:
  pgdata:
  redis_data:
  n8n_data:

Critical Environment Variables#

VariablePurposeDefaultProductionN8N_ENCRYPTION_KEYAES-256 key for credentialsAuto-generatedMust be set and shared across all nodesEXECUTIONS_MODE``regular or queue``regular``queue for productionDB_TYPEDatabase backendSQLitepostgresdb for queue modeWEBHOOK_URLPublic URL for webhookshttp://localhost:5678``https://n8n.example.com``EXECUTIONS_DATA_PRUNEAuto-delete old executionsfalse``true``EXECUTIONS_DATA_MAX_AGERetention in hours336 (14d)Set per policyN8N_GRACEFUL_SHUTDOWN_TIMEOUTWorker drain time (seconds)3060+ for long workflowsQUEUE_HEALTH_CHECK_ACTIVEWorker health endpointsfalse``true

⚠️ Queue Mode Requirements#

• PostgreSQL required — SQLite does NOT work in queue mode
• N8N_ENCRYPTION_KEY must be identical across main + all workers
• Community nodes must be installed on ALL containers (main + every worker)

Backup Strategy#

bash

# What to back up:
# 1. PostgreSQL database (pg_dump)
docker exec postgres pg_dump -U n8n n8n > n8n_backup_$(date +%Y%m%d).sql

# 2. n8n data volume (encryption key, config)
docker cp n8n:/home/node/.n8n ./n8n_data_backup/

# 3. .env file with passwords and encryption key

Critical: If you lose N8N_ENCRYPTION_KEY, ALL credentials become permanently unrecoverable. Back it up securely.

---

Credential Management#

Creating Credentials#

Credentials can be created via the REST API ( POST /credentials) but OAuth2 credentials require browser interaction for the consent flow.

bash

# Create an API key credential via API
curl -X POST "${N8N_BASE_URL}/api/v1/credentials" \
  -H "X-N8N-API-KEY: ${N8N_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My API Key",
    "type": "httpHeaderAuth",
    "data": {
      "name": "Authorization",
      "value": "Bearer sk-xxx"
    }
  }'

OAuth2 Credentials#

OAuth2 requires a browser redirect. Cannot be fully automated. Steps:

1. Create credential via API with clientId and clientSecret
2. User completes OAuth consent flow in browser
3. n8n stores the refresh/access tokens

Encryption Key Management#

• CRITICAL: N8N_ENCRYPTION_KEY encrypts ALL credentials with AES-256-GCM
• If lost, credentials are permanently unrecoverable — no reset possible
• Must be identical across main + all workers in queue mode
• Store in a password manager or .env file (never in docker-compose.yml in production)
• Rotating the key requires re-creating all credentials

Credential Types#

TypeAuth MethodAutomatable?httpHeaderAuthHeader-based (Bearer token)✅ YeshttpBasicAuthUsername + password✅ YesoAuth2ApiOAuth2 flow❌ Requires browsertelegramBotApiBot token✅ YespostgresApiConnection string✅ YesmysqlApiConnection string✅ YessmtpAccountSMTP credentials✅ YesslackApiBot token✅ Yes

---

Testing Strategy#

Pre-Deployment Checklist#

1. Validate structure — all nodes have name, type, valid connections
2. Check for dangerous patterns (see Safety Architecture)
3. Set pinData — mock input data for testing without real triggers
4. Deploy inactive — never activate a new workflow without testing
5. Manual trigger test — run with test data, verify output
6. Error branch test — intentionally trigger error paths
7. Activate — only after all tests pass

Mock Data with pinData#

json

{
  "nodes": [{
    "name": "Webhook",
    "type": "n8n-nodes-base.webhook",
    "parameters": { "path": "test-webhook" }
  }],
  "pinData": {
    "Webhook": [{ "json": { "email": "test@example.com", "subject": "Test" } }]
  }
}

pinData lets you test downstream nodes without the trigger actually firing.

Validation Command#

bash

python3 scripts/n8n_tester.py validate --file workflow.json --pretty

Test Execution#

bash

# Execute with test data
python3 scripts/n8n_api.py execute --id <workflow-id> --data '{"key": "value"}'

# Check result
python3 scripts/n8n_api.py get-execution --id <execution-id> --pretty

⚠️ Note: n8n has no native dry-run. execute_workflow runs the workflow for real. Always deploy workflows in inactive state and test with manual triggers before activating.

---

Webhook Patterns#

Incoming Webhook (External → n8n)#

text

External Service → POST https://n8n.example.com/webhook/my-path → n8n Workflow

Security options:

• none — no auth (⚠️ anyone can trigger)
• headerAuth — require specific header
• basicAuth — username + password
• jwtAuth — JWT token verification

Best practice: Always use at least headerAuth:

json

{
  "type": "n8n-nodes-base.webhook",
  "parameters": {
    "path": "my-secure-webhook",
    "authentication": "headerAuth",
    "headerAuth": {
      "name": "X-Webhook-Secret",
      "value": "={{ $env.WEBHOOK_SECRET }}"
    }
  }
}

Responding to Webhooks#

By default, n8n responds immediately with {"message": "Workflow was started"}. To send custom responses:

json

{
  "type": "n8n-nodes-base.respondToWebhook",
  "parameters": {
    "respondWith": "json",
    "responseBody": "={{ JSON.stringify({ status: 'ok', id: $json.id }) }}"
  }
}

Webhook URL Construction#

• Production: {WEBHOOK_URL}/webhook/{path}
• Test: {WEBHOOK_URL}/webhook-test/{path}
• Test webhooks only work when the workflow editor is open in the browser

---

Workflow Recipes#

Recipe 1: Email → AI Summarize → Telegram#

"The Executive Assistant"

text

📧 Email Trigger (IMAP)
  → 🧠 Code Node (extract sender, subject, body)
  → 🤖 OpenAI Node (summarize: "Summarize this email in 2 sentences")
  → 📱 Telegram Node (send summary)
  → 🏷️ Set Node (mark email as read)

Logic map: "When a new email arrives, extract the content, use AI to summarize it, send the summary to my Telegram, and mark the email as read."

Recipe 2: Webhook Receiver → Process → Respond#

"The API Processor"

text

🌐 Webhook (POST /process)
  → 🧪 Validate input (Code node: check required fields)
  → ↔️ IF node (valid input?)
    → ✅ Yes: Process data (HTTP Request to external API)
      → 📤 Respond to Webhook (success + result)
    → ❌ No: Respond to Webhook (400 + error message)
  → 🚨 Error Trigger → Log + notify

Logic map: "Receive a webhook, validate the input, process it if valid (or return an error), and respond to the caller."

Recipe 3: Scheduled Data Sync#

"The Data Synchronizer"

text

⏰ Schedule Trigger (every 6 hours)
  → 📥 HTTP Request (fetch from source API)
  → 🔀 Split In Batches (batch size: 100)
    → 🗃️ PostgreSQL Node (upsert records)
  → 🧮 Code Node (count synced records)
  → 📱 Telegram Node (sync summary: "Synced 847 records")
  → 🚨 Error Trigger → Log + retry notification

Logic map: "Every 6 hours, fetch data from the source API, batch-upsert into the database, and send me a summary on Telegram."

---

n8n API Gaps (What Requires UI)#

TaskAPI?WorkaroundCreate API key credentials✅POST /credentialsInstall community nodes❌npm install in container + restartOAuth2 consent flow❌Browser required — no automationUser management❌Internal API /rest/users or direct DBProject CRUD❌UI onlyWorkflow variables❌Use staticData in Code nodesLicense management❌UI only

---

Integration with OpenClaw Ecosystem#

Docker Pilot#

Delegate n8n deployment to Docker Pilot:

• n8n-pilot provides the Compose files
• Docker Pilot handles container lifecycle (start/stop/restart/health checks)
• Docker Pilot protects the n8n container as a critical service

Council of LLMs#

Use n8n webhooks to trigger Council deliberations:

text

n8n Webhook → HTTP Request to OpenClaw → Spawn Council → Return result → n8n continues

Thrift-Cycle#

n8n as the execution engine for periodic eBay data collection:

text

Schedule Trigger → Run Thrift-Cycle pipeline → Parse results → Alert on hot items

---

Quick Reference#

TaskCommand / PatternList workflowspython3 scripts/n8n_api.py list-workflows --prettyGet workflowpython3 scripts/n8n_api.py get-workflow --id ID --prettyCreate workflowpython3 scripts/n8n_api.py create --from-file workflow.jsonActivatepython3 scripts/n8n_api.py activate --id IDDeactivatepython3 scripts/n8n_api.py deactivate --id IDExecutepython3 scripts/n8n_api.py execute --id ID --data '{}'Validatepython3 scripts/n8n_tester.py validate --file workflow.json --prettyCheck executionspython3 scripts/n8n_api.py list-executions --limit 10 --prettyPerformancepython3 scripts/n8n_optimizer.py analyze --id ID --prettyCreate credentialcurl -X POST BASE_URL/api/v1/credentials -H "X-N8N-API-KEY: KEY"Backup DBdocker exec postgres pg_dump -U n8n n8n > backup.sqlInstall community nodedocker exec n8n npm install n8n-nodes-NAME && docker restart n8n

---

First-Run Setup#

When activating n8n-pilot on a new machine:

1. Detect n8n instance — check if port 5678 is listening, or scan Docker containers
2. Verify API access — test GET /api/v1/workflows with API key
3. Audit existing workflows — list all, check for failures, run dangerous pattern scan
4. Configure credentials — set up essential credentials (Telegram bot, email, database)
5. Deploy if missing — use Docker Pilot to deploy n8n with the self-hosting Compose files
6. Enable monitoring — set up health checks and execution pruning

---

Credits#

Extends the n8n skill by thomasansems (v2.0.0). This skill adds:

• ⚙️ Workflow design patterns (branching, merging, looping, sub-workflows)
• 🛡️ Dangerous pattern detection and safety architecture
• 🐳 Self-hosting guide (dev + production queue mode)
• 🔑 Credential management (creation, OAuth, encryption key)
• 🧪 Testing strategy (pinData, validation, pre-deployment checklist)
• 🔒 Webhook security patterns
• 📋 3 core workflow recipes
• 🚀 First-run setup and integration with Docker Pilot