OnionClaw — Tor / Dark Web OSINT#

v2.1.13 · by JacobJandon · MIT-0 License github.com/JacobJandon/OnionClaw

OnionClaw routes all requests through the Tor network. It queries 12 verified dark web search engines simultaneously, fetches .onion hidden-service pages, rotates Tor circuits, schedules recurring watch/alert jobs, and produces structured OSINT reports (Markdown, JSON, STIX, MISP, CSV) using the Robin investigation pipeline.

---

Setup (run once after install)#

bash

# 1. Install Python dependencies
pip3 install requests[socks] beautifulsoup4 python-dotenv stem

# 2. Interactive first-run wizard (sets up .env, torrc, and Tor in one step)
python3 {baseDir}/setup.py

# — OR — manual setup:
cp {baseDir}/.env.example {baseDir}/.env
# Edit {baseDir}/.env — add your LLM key (search + fetch work without one)

Start Tor (required before any command):

bash

# Linux:
sudo apt install tor && sudo systemctl start tor

# macOS:
brew install tor && brew services start tor

# Custom (no root needed — setup.py can do this automatically):
tor -f /tmp/sicry_tor.conf &
# torrc: SocksPort 9050 / ControlPort 9051 / CookieAuthentication 1 / DataDirectory /tmp/tor_data

Enable circuit rotation (required for renew.py and --daemon-poll):

text

Add to /etc/tor/torrc:
  ControlPort 9051
  CookieAuthentication 1
Then: systemctl restart tor
setup.py does this automatically.

---

Commands#

Check Tor is running#

Always run this first before any dark web operation.

bash

python3 {baseDir}/check_tor.py

Returns your exit IP and tor_active: true/false. If false, tell the user to start Tor before continuing.

---

Rotate Tor identity#

Get a fresh exit node and a new three-hop circuit. Use between sessions or whenever a new IP is needed.

bash

python3 {baseDir}/renew.py

Returns success: true/false. If false, ensure ControlPort 9051 is enabled and TOR_DATA_DIR is set in .env (or use setup.py).

---

Check which search engines are alive#

Ping all 12 engines via Tor and return latency + up/down for each.

bash

python3 {baseDir}/check_engines.py

Run before a large search session; pass the alive engine names to --engines to skip dead ones and save time.

---

Search the dark web#

Query all 12 dark web engines simultaneously. Returns deduplicated {title, url, engine} results.

bash

# Basic:
python3 {baseDir}/search.py --query "SEARCH_TERM"

# Limit results:
python3 {baseDir}/search.py --query "SEARCH_TERM" --max 30

# Specific engines:
python3 {baseDir}/search.py --query "SEARCH_TERM" --engines Ahmia Tor66 Ahmia-clearnet

Available engines: Ahmia, OnionLand, Amnesia, Torland, Excavator, Onionway, Tor66, OSS, Torgol, TheDeepSearches, DuckDuckGo-Tor, Ahmia-clearnet

Tip: Use short keyword queries (≤5 words). Dark web indexes respond far better to focused keywords than natural-language questions.

---

Fetch a .onion page#

Read the full text of any .onion URL (or clearnet URL) through Tor.

bash

python3 {baseDir}/fetch.py --url "http://SOME.onion/path"

Returns: {title, text (first 3000 chars), links, status, error}. If status: 0 or error is set, the hidden service is offline — they go down frequently; try a different result from search.py.

---

OSINT analysis#

Analyse raw dark web text with an LLM and produce a structured sectioned report.

bash

# From a string:
python3 {baseDir}/ask.py --query "QUERY" --mode MODE --content "RAW_TEXT"

# From a file:
python3 {baseDir}/ask.py --query "QUERY" --mode MODE --file /path/to/content.txt

# From stdin (pipe):
echo "CONTENT" | python3 {baseDir}/ask.py --query "QUERY" --mode MODE

Analysis modes:

ModeUse forthreat_intelGeneral OSINT (default) — artifacts, insights, next stepsransomwareMalware / C2 / MITRE ATT&CK TTPs, victim orgs, indicatorspersonal_identityPII / breach exposure, severity, protective actionscorporateLeaked credentials / code / internal docs, IR stepsbash

# With custom focus appended to the prompt:
python3 {baseDir}/ask.py --query "QUERY" --mode threat_intel \
  --custom "Focus on cryptocurrency wallet addresses"

---

Full OSINT pipeline (single command)#

Runs the complete Robin pipeline: refine query → check live engines → search → filter best results → batch scrape → OSINT analysis → save report

bash

python3 {baseDir}/pipeline.py --query "INVESTIGATION_QUERY" --mode MODE

Essential flags:

FlagDefaultDescription--query TEXTrequiredInvestigation topic (natural language OK — refined automatically)--mode MODE``threat_intel``threat_intel / ransomware / personal_identity / corporate``--max N``30Max raw results from search--scrape N``8Pages to batch-fetch (use 0 to skip scraping and get results-only report)--custom TEXTExtra LLM instructions appended to the mode prompt--out FILESave report to file (exits 1 on permission error)--format FMT``mdOutput format: md / json / csv / stix / misp``--no-llmSkip all LLM steps — dump raw results / entity extraction only--confidenceShow BM25 confidence score per result--engines NAME…Restrict to specific engines (skip dead ones)--no-cacheBypass query/page cache for this run--clear-cacheFlush the result cache, then run--resume JOB_IDResume a checkpointed pipeline run by job ID--interactiveAfter the report, open a follow-up REPL for drill-down--output-dir DIRWrite <job_id>.<ext> into DIR (batch pipeline friendly)--modesList all modes and their engine routing, then exit--engine-statsPrint per-engine reliability / latency table, then exit--check-updateCheck for a newer OnionClaw release and exit--versionPrint version and exit

MISP-specific flags:

FlagDefaultDescription--misp-threat-level N``2MISP threat level 1–4 (1=high, 4=undefined)--misp-distribution N``0MISP distribution (0=your org, 1=connected, 2=all, 3=inherited)

Watch / alert flags:

FlagDescription--watchRegister this query as a recurring watch job and exit--interval HOURSRe-run interval in hours for --watch (default 6)--watch-checkRun all due watch jobs now and print alerts--watch-check --output-dir DIRSame but write each job's JSON to DIR (exits 1 on write error)--watch-listList all active watch jobs--watch-disable JOB_IDDisable a watch job by ID--watch-clear-allDisable ALL active watch jobs at once--watch-daemon(deprecated alias) Run as a blocking daemon loop--daemon-poll SECONDSRun --watch-check every N seconds in a daemon loop

---

Daemon mode (continuous monitoring)#

Keep OnionClaw running and poll watch jobs at a fixed interval:

bash

python3 {baseDir}/pipeline.py --daemon-poll 3600   # check every hour

---

Scheduling watch jobs#

Register a query as a recurring alert:

bash

# Register (runs every 6 hours by default):
python3 {baseDir}/pipeline.py --query "ransomware hospital 2026" --watch --interval 6

# List all active jobs:
python3 {baseDir}/pipeline.py --watch-list

# Check due jobs now and write JSON files for each:
python3 {baseDir}/pipeline.py --watch-check --output-dir /tmp/alerts/

# Disable one job:
python3 {baseDir}/pipeline.py --watch-disable <JOB_ID>

# Clear all:
python3 {baseDir}/pipeline.py --watch-clear-all

---

Typical investigation flows#

"Search the dark web for X"#

1. python3 {baseDir}/check_tor.py — verify connected
2. python3 {baseDir}/search.py --query "X" — search all 12 engines
3. python3 {baseDir}/fetch.py --url "URL" — read top 2–3 results
4. python3 {baseDir}/ask.py --mode threat_intel --query "X" --content "..." — generate report

"Has company.com appeared in dark web leaks?"#

1. python3 {baseDir}/check_tor.py
2. python3 {baseDir}/pipeline.py --query "company.com credentials leak" --mode corporate
3. Present the structured report

"Investigate ransomware group X"#

1. python3 {baseDir}/check_tor.py
2. python3 {baseDir}/pipeline.py --query "GROUP_NAME ransomware" --mode ransomware

"Write a STIX bundle for this investigation"#

bash

python3 {baseDir}/pipeline.py \
  --query "QUERY" --mode threat_intel \
  --format stix --out bundle.json

"Fetch this .onion URL"#

1. python3 {baseDir}/check_tor.py
2. python3 {baseDir}/fetch.py --url "URL"
3. Show the user the title + text content

"Monitor for new leaks mentioning acme.com, alert me daily"#

bash

python3 {baseDir}/pipeline.py \
  --query "acme.com leak credentials" --watch --interval 24
# Later, in a cron job or daemon:
python3 {baseDir}/pipeline.py --watch-check --output-dir /tmp/acme-alerts/

---

Output formats#

FormatFlagUse forMarkdown--format md (default)Human-readable reports, --out report.mdJSON--format jsonStructured machine-readable, automationCSV--format csvSpreadsheet import, result listsSTIX 2.1--format stixThreat-intel platforms (MISP, OpenCTI, Splunk ES)MISP--format mispDirect MISP event import

---

Important notes#

• All traffic routes through Tor — tell the user this when relevant.
• .onion hidden services go offline frequently. status: 0 means the site is temporarily unreachable — try a different result from search.py.
• Dark web search indexes go down often — run check_engines.py first and pass only alive engine names with --engines.
• LLM tools ( ask.py, pipeline steps 3/5/7) require an API key in {baseDir}/.env. Set LLM_PROVIDER=ollama for fully local inference with no key. search.py, fetch.py, check_tor.py, renew.py, and check_engines.py work with no key at all.
• --scrape 0 skips page fetching. The pipeline still runs step 7 (LLM analysis on search-result metadata only) and writes --out / --output-dir normally. A WARN: --scrape 0 notice is printed to stderr.
• Use responsibly and lawfully — OSINT, security research, and threat intelligence only.

---

Maintenance#

Update the bundled sicry.py engine#

OnionClaw bundles sicry.py from the upstream SICRY™ repo. After a new SICRY™ release, sync the bundled copy:

bash

# Pull latest:
python3 {baseDir}/sync_sicry.py

# Pull a specific release tag:
python3 {baseDir}/sync_sicry.py --tag v2.1.13

# Preview without writing:
python3 {baseDir}/sync_sicry.py --dry-run

Checking for OnionClaw updates#

OnionClaw checks the GitHub Releases API (published releases only — not plain git tags) for newer versions. A one-line notice is printed automatically at pipeline startup when an update is available.

bash

# On-demand update check:
python3 {baseDir}/pipeline.py --check-update

# Programmatic:
import sicry
r = sicry.check_update()
if not r["up_to_date"]:
    print(f"Update: {r['current']} → {r['latest']}  {r['url']}")

bash

# Upgrade:
git -C {baseDir} pull
python3 {baseDir}/sync_sicry.py