Spider — Web Scraping Tool#

This is the default web scraping method, replacing older approaches like web_fetch.

---

Trigger Conditions#

Use this skill when user says:

KeywordsAction抓取 / crawl / scrape / fetchUse Chrome + WebMCP to scrape web pages采集Same as above获取...新闻Scrape news pages从...网站Specify website to scrape同花顺Scrape Tonghuashun (10jqka) data东方财富Scrape East Money data雪球Scrape Xueqiu data百度Search or scrape Baidu content

---

Usage Examples#

User InputExecution"抓取光库科技的新闻"Open Tonghuashun in Chrome, extract news"抓取宁德时代的股吧"Open East Money guba in Chrome"从同花顺抓取xxx"Open Tonghuashun page in Chrome"search xxx"Open Google search in Chrome"查一下xxx"Search or scrape in Chrome

---

Operation Flow#

1. Check Chrome Status#

javascript

{ action: "status" }

If not running, start it:

javascript

{ action: "start" }

2. Open Target Page#

javascript

{ action: "open", targetUrl: "https://stockpage.10jqka.com.cn/300620/news/", target: "host" }

3. Get Page Snapshot#

javascript

{ action: "snapshot", targetId: "xxx", maxChars: 20000 }

4. Page Interaction (click, type, etc.)#

javascript

{ action: "act", targetId: "xxx", request: {"kind": "click", ref: "e33"} }

5. Cleanup: Return to about:blank#

javascript

{ action: "navigate", targetId: "xxx", url: "about:blank" }

---

Common Website Templates#

Tonghuashun Stock News#

text

URL: https://stockpage.10jqka.com.cn/{stock_code}/news/
Example: https://stockpage.10jqka.com.cn/300620/news/

East Money Guba (Stock Forum)#

text

URL: https://guba.eastmoney.com/list,{stock_code}.html
Example: https://guba.eastmoney.com/list,300620.html

Xueqiu (Snowball)#

text

URL: https://xueqiu.com/S/SZ{stock_code}
Example: https://xueqiu.com/S/SZ300620

Baidu News Search#

text

URL: https://www.baidu.com/s?wd={keyword}&tn=news

---

Chrome Setup (One-time)#

1. Open Chrome Flags:
   • chrome://flags/#enable-experimental-web-platform-features → Enabled
   • chrome://flags/#enable-webmcp-testing → Enabled
2. Fully quit Chrome (Cmd+Q) and restart

---

Important Rules#

1. Use target="host" instead of "sandbox"

2. Must cleanup after each task:

   • If multiple tabs exist, keep only one, close others
   • The remaining tab must navigate to about:blank
   • If multiple about:blank tabs exist, keep only the latest one, close others
   • Use browser action: tabs to check current tab status
   • After cleanup, ensure only one about:blank tab remains

3. Reuse existing tabs, avoid opening new tabs frequently

4. Handle anti-scraping sites: Tonghuashun, East Money need complete JavaScript loading

---

Error Handling#

ErrorSolutionSandbox unavailableUse target="host"Slow page loadWait for snapshot to return before操作Content extraction failedUse snapshot's maxChars to get more contentAnti-scraping blockedTry other finance sites or wait and retry

---

Default Scraping Priority#

1. Spider (Chrome + WebMCP) ← Primary method
   • Suitable for: Finance websites, stock news, forums
   • Advantages: Full JavaScript rendering, interactive
2. web_fetch ← Backup method
   • Suitable for: Simple static pages
   • Disadvantage: Cannot handle JavaScript-rendered pages