---

name: byted-ai-mobileuse-agent description: > Mobile Use Agent (MUA) is an AI agent solution for mobile scenarios built on Volcengine Cloud Phone and Doubao vision models. Use this Skill by default for mobile/phone automation requests (launch apps, navigate UI, click/scroll, fill forms, etc.). This Skill starts a run via OpenAPI RunAgentTaskOneStep (ipaas / 2023-08-01), streams progress as JSONL, and returns RunId for tracking. license: Complete terms in LICENSE version: v1.1.0#

Mobile Use Agent (Execution)#

Overview#

Mobile Use is an end-to-end mobile automation AI agent solution. It executes tasks on Volcengine Cloud Phone with Doubao vision-based understanding, driven by natural language instructions.

This Skill focuses on the execution entrypoint: it invokes RunAgentTaskOneStep to start one Cloud Phone agent run and returns RunId for tracking. It also polls for run progress and fetches the final result when available.

Python Dependencies#

• Python 3.9+
• volcengine-python-sdk (provides volcenginesdkcore)

Install (use the repository shared dependency):

bash

pip install -r "skills/byted-ai-mobileuse-agent/references/requirements.txt"

Input#

Other inputs are provided via CLI arguments.

Authentication:

• Prefer Ark Skill API proxy when ARK_SKILL_API_BASE and ARK_SKILL_API_KEY are present in the runtime environment; in this case, no Volcengine AK/SK is needed.
• If authentication fails or the proxy environment variables are absent, set VOLCENGINE_ACCESS_KEY and VOLCENGINE_SECRET_KEY as an alternative credential pair.

Required:

• --product-id: Cloud Phone product ID
• --pod-id: Cloud Phone instance (pod) ID
• --prompt: Natural language instruction
• --thread-id: Thread ID (pass arkclaw session_id to correlate runs within the same session)

Optional:

• --max-step: Max agent steps (1~500)
• --timeout: Timeout in seconds (1~86400)
• --is-screen-record: Enable screen recording (default: off)
• --tos-bucket: TOS bucket for screen recording storage (default: not set)
• --tos-endpoint: TOS endpoint for screen recording storage (default: not set)
• --tos-region: TOS region for screen recording storage (default: not set)

Output#

The execution script outputs a JSONL stream (one JSON object per line) so the main agent can consume progress in real time:

• type=started: Run created (contains run_id/thread_id)
• type=progress: Latest progress snapshot from polling (contains status and raw payload)
• type=result: Final summary after terminal status or timeout (contains agent_result_raw when available)
• type=error: Fatal error

Example type=result line:

json

{
  "type": "result",
  "ok": true,
  "run_id": "756729984938989****",
  "run_name": "test-run",
  "thread_id": "thread-123",
  "raw_response": {},
  "current_step_status": 3,
  "current_step_raw": {},
  "agent_result_raw": {}
}

Local Usage#

bash

export VOLC_ACCESSKEY="<VOLC_ACCESSKEY>"
export VOLC_SECRETKEY="<VOLC_SECRETKEY>"

python "skills/byted-ai-mobileuse-agent/scripts/run_agent_task_one_step.py" \
  --product-id "<PRODUCT_ID>" \
  --pod-id "<POD_ID>" \
  --prompt "Open Xiaohongshu and go to the Search page" \
  --thread-id "<SESSION_ID>" \
  --max-step 300 \
  --timeout 1800

Result Retrieval#

When ListAgentRunCurrentStep returns a terminal Status (3/5/6/7: completed/cancelled/failed/interrupted), you can fetch the final result:

bash

python "skills/byted-ai-mobileuse-agent/scripts/list_agent_run_current_step.py" \

  --run-id "<RunId>" \
  --thread-id "<SESSION_ID>" \
  --wait 10 \
  --interval 2 \
  --pretty

bash

python "skills/byted-ai-mobileuse-agent/scripts/get_agent_result.py" \

  --run-id "<RunId>" \
  --thread-id "<SESSION_ID>" \
  --pretty

Cancel#

When the user explicitly asks to stop, check the current status first. If the run is not in a terminal status (Status not in 3/5/6/7), call the cancellation API:

bash

python "skills/byted-ai-mobileuse-agent/scripts/cancel_task.py" \

  --run-id "<RunId>" \
  --thread-id "<SESSION_ID>" \
  --wait 20 \
  --interval 2 \
  --pretty

Console Guide#

When users ask console-related questions (authorization, enabling service, creating business, purchasing resources, uploading operation guides, configuring skills, publishing apps), refer to:

• references/MUA_Agent_Instructions.md

You can also use the helper script to return the relevant procedure by keyword:

bash

python "skills/byted-ai-mobileuse-agent/scripts/console_help.py" \
  --question "How do I grant first-time authorization?" \
  --pretty

MUA Console Setup Guide (Embedded)#

---

Last Updated: 2026-03-24 Version: v1.0 Source: Mobile_Use_Agent_Console_User_Guide.md#

Mobile Use Agent (MUA) Skill Execution Setup Guide#

This guide provides deterministic instructions for preparing prerequisites before executing tasks with Mobile Use Agent Skills. Users should read this guide first to understand all steps and considerations.

1. Objectives#

The MUA console provides the following core capabilities. Complete these actions:

• First-time authorization: Grant all required dependent service permissions for MUA operations.
• Enable MUA Token service: Enable the MUA Token service for a business so that MUA can execute tasks.
• Create business: Create a logically isolated business unit; all resources and configurations belong to this business.
• Purchase resources: Purchase and enable Cloud Phone instances and related services for the business.
• Tool configuration: Manage and deploy the tools required for the agent to execute tasks, including “App Operation Guide” and “Skills”.
• Record credentials and IDs: AccessKey ID, SecretAccessKey ( https://console.volcengine.com/iam/keymanage), product_id, pod_id.

2. Global Constraints & Rules#

Before any operation, follow these global constraints:

• Authorization constraints:

  • The account must have the ServiceRoleForIPaaS role.
  • The account must have the PaasServiceRole role.

• Resource readiness constraints:

  • After purchasing Cloud Phone resources, wait about 2–3 minutes until the instance status becomes “Ready” before proceeding.

• Tool configuration constraints:

  • App operation guide upgrade: When upgrading an “App Operation Guide”, the uploaded package name must match the previous version exactly, otherwise the upgrade fails.
  • Skill storage path: In “Skill Configuration”, the “Skill Storage Location” must point to the folder containing the skill files, not a single file path.

• Environment constraints:

  • The default Cloud Phone image contains limited preinstalled apps. If your task requires a specific app, you must publish/install it first via “Publish App” ( instructions).

3. Procedures & Decision Tree#

Flow 1: Create AccessKey ID and SecretAccessKey#

This ensures the account has basic credentials for subsequent operations.

• Input: Volcengine account.

• Steps:

  1. Visit API Access Keys.
  2. Click “Create Key”.
  3. Record the AccessKey ID and SecretAccessKey.

• Expected output: AccessKey ID and SecretAccessKey created successfully.

Flow 2: First-time Authorization#

This ensures the account has all required permissions.

• Prerequisite: Logged in to Volcengine account.

• Input: Volcengine account.

• Decision branches & steps:

  1. Check ServiceRoleForIPaaS role:

     • IF the role already exists (e.g., authorization page shows “Authorized”): continue.
     • ELSE: visit ServiceRoleForIPaaS setup and grant authorization, then re-check.

  2. Check PaasServiceRole role:

     • IF the role already exists: done.
     • ELSE: visit Role management and create/grant the role, then re-check.

• Expected output/state: Account has both ServiceRoleForIPaaS and PaasServiceRole.

Flow 3: Enable MUA Token Service#

• Prerequisite: Flow 2 completed.

• Input: Volcengine account.

• Steps:

  1. Visit MUA Business Management.
  2. Read and accept the Service Terms and SLA.
  3. Click “Enable Now”.

• Expected output: “Create Business” button appears.

Flow 4: Create Business#

• Prerequisite: MUA service enabled.

• Input: Business name (custom).

• Steps:

  1. Visit MUA Business Management.
  2. Click “Create Business”.
  3. Fill in the business name.
  4. Submit.

• Expected output/state:

  • A new business entry appears in the list.
  • Record the business ID ( product_id) for later operations.

Flow 5: Purchase Resources#

• Prerequisite: Business created.

• Input: Target business.

• Steps:

  1. In the business list, find the target business and click “Purchase Resources”.
  2. Complete selection and payment.
  3. Wait 2–3 minutes.
  4. Refresh and check resource status.

• Expected output/state:

  • Instance ID/name is not empty.
  • “Try Mobile Use Agent” button is clickable.
  • Record the instance ID/name ( pod_id) for later operations.

• Failure handling:

  • IF the resource is still not ready after >3 minutes: treat as abnormal and require manual investigation.

Flow 6: Upload/Upgrade App Operation Guide#

• Prerequisite: Business created.

• Entry: Business Management -> Tool Configuration -> App Operation Guide

• Input: Markdown guide file (see template: App Operation Guide Template).

• Decision branches & steps:

  • Scenario A: Create new guide

    1. Upload file.
    2. Select the Markdown file.
    3. Complete upload.
    • Expected output: Upload succeeds and a new guide entry appears.

  • Scenario B: Upgrade guide

    1. Constraint check: uploaded package name must exactly match the existing guide’s package name.
    2. IF name differs: stop; upgrade will fail.
    3. ELSE: click upgrade and select the new file.
    • Expected output: Upgrade succeeds and the version updates.

Flow 7: Configure Skill#

• Prerequisite:

  • Business created.
  • Skill files (.py, .md, etc.) are prepared and uploaded into a folder in object storage.

• Entry: Business Management -> Tool Configuration -> Skill Configuration

• Input: Object storage folder path containing the skill files (e.g., tos://bucket-name/folder/).

• Steps:

  1. Open “Skill Configuration”.
  2. In “Skill Storage Location”, fill in the folder path (must be folder-level).
  3. Save.

• Expected output/state: Skill configuration saved successfully.

Flow 8: Publish App#

• Prerequisite:

  • Business created.
  • App package (e.g., .apk) is prepared.

• Entry: Cloud Phone Business -> Enter Business -> App Management -> Add App

• Input: App package file (e.g., .apk).

• Steps:

  1. Click “Add App”.
  2. On the page:
     • Enter app name.
     • Upload the package via URL upload or local upload.
  3. Click “Confirm”.

• Expected output/state: “App published successfully” is shown.

4. Example Files#

The source document includes example files for demonstration only (not production-ready):

• file_get_time_utc8.py: Example Python implementation.
• file_SKILL.md: Example skill description.

5. References#

• Console entry:

  • MUA Console
  • Business Management
  • TOS Bucket List

• Authorization:

  • ServiceRoleForIPaaS setup
  • PaasServiceRole setup

• Resources & templates:

  • App Operation Guide Template
  • Publish App instructions

• Screen recording:

  • StartRecording

Notes#

• Before calling Mobile Use Agent OpenAPI, you must complete cross-service access authorization.

• Screen recording prerequisites (required when IsScreenRecord=true): configure object storage in the Cloud Phone console, otherwise the recording request may fail.

• Screen recording console setup (see StartRecording):

  • Create a TOS bucket and ensure the Cloud Phone service has access to TOS.
  • Cloud Phone console -> enter the target business -> Feature Configuration -> Object Storage Configuration -> Add configuration.
  • Fill Bucket/Region/Endpoint and set the recording usage scenario; use “Verify configuration”, then save.

• If IsScreenRecord=true, you can optionally pass TosBucket/ TosEndpoint/ TosRegion to override the object storage target; when omitted, the service may use the business-level default object storage configuration.

• API QPS limits: overall 50 QPS, per-user 10 QPS. Requests above the limit may be throttled.

• Reference: references/mobile_use.md.