Log Parser#

A log parsing and analysis tool for security operations and DevOps.

Features#

• Multi-format Support - nginx, apache, syslog, application logs
• Auto-detection - Automatically detect log format
• Key Extraction - Extract IPs, timestamps, error codes, URLs
• Filtering - Filter logs by IP, status code, time range
• Statistics - Generate summary reports
• Error Detection - Identify and highlight error entries

Usage#

bash

python3 skills/log-parser/scripts/log_parser.py <action> [options]

Actions#

ActionDescriptionparseParse log file and extract fieldsstatsGenerate statistics reportfilterFilter logs by criteriaerrorsExtract error entries onlytopTop N items (IPs, URLs, etc.)

Options#

OptionTypeDefaultDescription--filestring-Log file path--formatstringautoLog format (auto, nginx, apache, syslog)--limitint100Max results to return--filter-ipstring-Filter by IP address--filter-statusstring-Filter by status code--top-fieldstring-Field for top N (ip, url, status)

Supported Log Formats#

nginx#

text

192.168.1.1 - - [22/Mar/2026:14:00:00 +0800] "GET /api/test HTTP/1.1" 200 1234

apache#

text

192.168.1.1 - - [22/Mar/2026:14:00:00 +0800] "GET /api/test HTTP/1.1" 200 1234

syslog#

text

Mar 22 14:00:00 server sshd[12345]: Failed password for root from 192.168.1.1

application logs (JSON)#

json

{"timestamp": "2026-03-22T14:00:00Z", "level": "ERROR", "message": "..."}

Examples#

bash

# Parse log file
python3 skills/log-parser/scripts/log_parser.py parse --file /var/log/nginx/access.log

# Generate statistics
python3 skills/log-parser/scripts/log_parser.py stats --file /var/log/nginx/access.log

# Filter by IP
python3 skills/log-parser/scripts/log_parser.py filter --file /var/log/nginx/access.log --filter-ip 192.168.1.1

# Get top 10 IPs
python3 skills/log-parser/scripts/log_parser.py top --file /var/log/nginx/access.log --top-field ip --limit 10

# Extract errors
python3 skills/log-parser/scripts/log_parser.py errors --file /var/log/nginx/access.log

Use Cases#

1. Security Analysis - Identify suspicious IPs, failed logins
2. Performance Monitoring - Find slow requests, errors
3. Traffic Analysis - Top URLs, user agents
4. Debugging - Extract error entries quickly

Output Format#

All results are returned in JSON format:

json

{
  "success": true,
  "total": 1000,
  "parsed": 998,
  "entries": [...],
  "stats": {...}
}

Current Status#

In development.