smart-research

smart-research#

Universal research skill — multi-engine search + multi-layer fallback fetching + structured results.

Features#

• 🔍 Multi-Engine Search: Baidu, DuckDuckGo, Bing with automatic fallback
• 🔄 Multi-Layer Fallback Fetching: crawl4ai → Jina → markdown.new → defuddle → Playwright
• 📊 Structured Output: title / body / href / source_type / fetch_method / fetched_at
• 🚀 Zero API Key: All services are free
• 📦 One-Click Research: Single research() action for search + fetch + fusion

Architecture#

text

┌─────────────────────────────────────────────┐
│           smart-research 统一入口            │
└─────────────────────────────────────────────┘
                     │
   ┌─────────────────┼─────────────────┐
   ▼                 ▼                 ▼
┌────────┐      ┌────────┐       ┌────────┐
│ Search │      │ Fetch  │       │ Fusion │
│ Layer  │      │ Layer  │       │ Layer  │
└────────┘      └────────┘       └────────┘

Usage#

Python API#

python

result = main({
    "action": "research",
    "query": "Python tutorial",
    "num_results": 5,
    "crawl_depth": 3
})

Search Only#

python

result = main({
    "action": "search",
    "query": "machine learning latest research",
    "num_results": 5
})

Fetch Only#

python

result = main({
    "action": "fetch",
    "url": "https://example.com/article"
})

Deep Search#

python

result = main({
    "action": "deep_search",
    "query": "AI agent trends 2024",
    "num_results": 5
})

Input Parameters#

ParameterTypeRequiredDefaultDescriptionactionstringYes-Operation type: research, search, fetch, deep_search``querystringConditional-Search query (required for research/ search/ deep_search)urlstringConditional-Target URL (required for fetch)num_resultsintNo5Number of search results (1-20)crawl_depthintNo3Number of top results to fetch details (1-10)

Output Format#

Research Result#

json

{
  "success": true,
  "query": "Python tutorial",
  "search_results": [
    {
      "title": "Result title",
      "href": "https://example.com",
      "body": "Snippet content",
      "score": 85.5,
      "source_type": "baidu",
      "fetch_method": "crawl4ai",
      "fetched_at": "2024-01-01T12:00:00Z"
    }
  ],
  "message": "Research completed"
}

Fetch Result#

json

{
  "success": true,
  "url": "https://example.com",
  "content": "# Article Title\n\nClean markdown content...",
  "source": "jina",
  "fetched_at": "2024-01-01T12:00:00Z"
}

Execution#

yaml

type: script
script_path: scripts/smart_research.py
entry_point: main
dependencies:
  - uv>=0.1.0
  - requests>=2.28.0
  - baidusearch>=1.0.3
  - crawl4ai>=0.8.0
  - playwright>=1.40.0

Fetch Fallback Chain#

Each URL automatically tries:

PriorityServiceTimeoutDescription1crawl4ai15sAI-powered, local2Jina Reader10sFree, no key needed3markdown.new8sSimple pages4defuddle8sBetter noise reduction5Playwright30sDynamic rendering

Privacy Notice#

• Public web pages: Fully supported
• Private/internal URLs: Not supported by default
• Sensitive content: Requires user consent

Error Handling#

• Returns {"success": false, "message": "..."} on errors
• Automatically falls back to next service
• Partial results returned if some fetches fail