ByteDance Visual Recognition — Doubao-Seed + GLM#

Doubao-Seed (6 models) + Zhipu GLM (2 free models). First run auto-generates config.json, fill in one API Key to start. IAM console usage syncs automatically on each recognition.

Privacy & Data Notice#

• Network: Selected images/videos and prompts are base64-encoded and sent to Volcengine (Doubao) or Zhipu (GLM) cloud APIs.
• Local cache: Media files temporarily copied to Temp/YYYYMMDD/, default 7-day retention ( temp_retention_days in config.json, range 1-3650).
• History: Recognition history stored in vision_history.json, follow-up context in .last_response, auto-cleaned after 7 days. GLM follow-up reuses full message history including base64 media data; Doubao follow-up uses previous_response_id without re-transmitting files.
• Credentials: API Keys stored in plaintext config.json. Do not use personal keys on shared machines.
• IAM sync: Automatically syncs token usage from Volcengine IAM console on each recognition when IAM credentials are configured.
• First run: A privacy notice is displayed once. Continuing past it acknowledges data handling practices.

Setup (pick one)#

Doubao (Volcengine)#

1. Join the Collaboration Rewards Program for free quota, then get your API Key
2. Create inference endpoints, pick from:

config keymodelprioritydoubao_seed_21p_idDoubao-Seed-2.1-Proprimarydoubao_seed_21t_idDoubao-Seed-2.1-Turbosecondarydoubao_seed_20p_idDoubao-Seed-2.0-Protertiarydoubao_seed_20c_idDoubao-Seed-2.0-Codecode-firstdoubao_seed_20l_idDoubao-Seed-2.0-Litefallbackdoubao_seed_20m_idDoubao-Seed-2.0-Minilow-cost

1. Edit config.json, replace "" with actual endpoint IDs.

GLM (Zhipu, free)#

1. https://open.bigmodel.cn → get API Key
2. Edit config.json: "zhipu_api_key": "your-key"

Provider filter#

provider_mode in config.json:

• 0 = all (default)
• 1 = Zhipu only
• 2 = Doubao only

Test#

bash

python doubao_vision_recognize.py --help
python doubao_vision_recognize.py status

---

Commands#

commandpurposeexamplerec <file> --image|--video --text|--coderecognizerec a.jpg --image --text``rec <dir> --batch --image|--video --text|--codebatchrec ./img/ --batch --image --text``ask --text|--code --prompt "..."follow-upask --text -p "details"``statususage statssyncmanual console synchistory7-day history

Parameters#

paramdesc--imageimage input--videovideo input--texttext output--codecode output--prompt / -pextra instruction--batchdirectory batch

---

Behavior Rules#

1. Trigger only on listed patterns#

Activate only when the user's message matches one of the trigger_patterns above. Do NOT activate on loosely related text. If uncertain, ask before executing.

2. Parameter inference#

• "recognize/analyze image" → --image --text
• "recognize/analyze video" → --video --text
• "convert to code / UI to code / design to code" → --code
• extra requirements → --prompt "..."
• unsure → ask once if image or video

3. Credential safety#

API keys are stored in plaintext config.json. Warn users not to use high-value keys on shared machines. First-run privacy notice already discloses plaintext storage.

---

Limits#

• Doubao: 180W tokens per model per day, auto-fallback
• GLM: free, auto-retry on failure (4.6V: 10 retries, 4.1V: 5 retries)
• Image ≤ 15MB, Video ≤ 50MB