CADT Alibaba Cloud Resource Probe#

Use the CADT probe service (BPStudio API) to discover cloud resources under an Alibaba Cloud account.

Quick Start#

Prerequisites#

bash

pip install alibabacloud_bpstudio20210931==8.1.1 alibabacloud_credentials==1.0.5

Configure credentials using the Alibaba Cloud default credential chain. Choose one of the following methods:

• Run aliyun configure to set up a credential profile
• Set environment variables ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET (auto-detected by the default chain)
• Use an ECS instance RAM role (when running on ECS)
• Configure ~/.alibabacloud/credentials file

Pre-conditions#

The Cloud Config service must be enabled in the Alibaba Cloud console, as CADT probe relies on resource data provided by Cloud Config:

• Enable at: https://config.console.aliyun.com/
• Monitoring scope must be set to "All resources in account"
• After initial activation, wait 30-60 minutes for the initial scan to complete

RAM Policy#

This skill requires the following RAM permissions to execute BPStudio probe operations and discover cloud resources across 20+ Alibaba Cloud services.

For the complete permission list including a least-privilege RAM policy JSON, see references/ram-policies.md.

Key permissions:

ServiceActionPurposeBPStudiobpstudio:ExecuteOperationExecute all probe sync operations (CreateOneClickJob, GetOneClickStatus, GetProbeResourceSummary, GetProbeRelatedItems, GetLastProbeTime, GetProbeRegions, GetProbeResult)Configconfig:ListDiscoveredResources + 5 othersResource discovery via Cloud ConfigVPC / SLB / ALB / NLB / ACK / ...Various Describe/List actionsProduct-specific resource discovery and relation queries

[MUST] Permission Failure Handling: When any command or API call fails due to permission errors at any point during execution, follow this process:

1. Read references/ram-policies.md to get the full list of permissions required by this SKILL
2. Use ram-permission-diagnose skill to guide the user through requesting the necessary permissions
3. Pause and wait until the user confirms that the required permissions have been granted

Run Probe#

bash

python scripts/probe.py

Parameters#

ParameterRequiredDescription--regionsNoComma-separated list of regions to probe and filter results (e.g., cn-hangzhou,cn-beijing). Probes all regions when not specified--outputNoOutput JSON file path. Prints to console only when not specified--list-typesNoComma-separated resource type filter list (serviceTypes) for GetProbeResult / GetProbeResourceSummary; empty means no filter--skip-probeNoSkip task creation and polling, directly pull existing probe results (suitable for previously probed scenarios)-q, --quietNoQuiet mode: suppress console decorative output, only output pure JSON to stdout (suitable for Agent/script calls). [debug] output is disabled by default in this mode--debugNoEnable debug output: print [debug] details of API responses (with full response body) to stderr. Disabled by default in quiet mode (-q)--summaryNoSummary mode: only output aggregated statistics (skip GetProbeResult detailed resource list), significantly reducing output size--keep-attributesNoRetain the original attributes field for each resource (stripped by default to reduce output size)--related-resource-idNoResource ID for querying related resources (GetProbeRelatedItems). When specified, skips full probe and only queries related resources for this resource (e.g., i-xxx for ECS, sg-xxx for security group, vpc-xxx for VPC)--related-typesNoComma-separated resource type filter for --related-resource-id (e.g., security_group,vswitch). Empty means return all related resource types

Usage Examples#

1. Basic Probe#

bash

python scripts/probe.py

2. Export JSON Results#

bash

python scripts/probe.py --output probe_result.json

3. Skip Probe and Pull Existing Results#

If you have previously probed (resource data is cached on the server side), you can directly pull existing results without creating a new task:

bash

python scripts/probe.py --skip-probe
python scripts/probe.py --skip-probe --output result.json
python scripts/probe.py --skip-probe --list-types ecs,vpc

4. Agent Mode (Quiet Mode)#

Use the -q flag to suppress all human-readable output, printing only pure JSON to stdout for easy Agent or script parsing. By default, the attributes field of each resource is stripped to reduce output size:

bash

python scripts/probe.py -q
python scripts/probe.py -q --output result.json
python scripts/probe.py -q --regions cn-hangzhou --list-types ecs,vpc

To retain full attributes, use --keep-attributes:

bash

python scripts/probe.py -q --keep-attributes

Output Auto-Degradation (Agent Mode)

When the JSON output is large (e.g., 127 security groups with Permissions arrays + 100 vswitches), the script automatically degrades output to prevent terminal truncation:

Output SizeBehavior≤ 50KBNormal JSON (with indentation) to stdout50KB ~ 200KBCompact JSON (no indentation) to stdout, warning on stderr> 200KBFull data written to file, stdout outputs a summary JSON only

When output exceeds 200KB, stdout returns a summary like:

json

{
  "status": "output_too_large",
  "output_file": "/path/to/file.json",
  "total_resources": 246,
  "by_type": {"security_group": 127, "vswitch": 100, "ecs": 19},
  "hint": "Read the output file for full results"
}

Recommendation: Always use --output to write full results to a file, then read the file for parsing. This avoids any output size issues:

bash

python scripts/probe.py -q --output result.json

5. Summary Mode (Aggregated Statistics Only)#

When there are many resources, use --summary to skip the detailed resource list and only output aggregated statistics by region/type, suitable for dashboard analysis:

bash

python scripts/probe.py --summary
python scripts/probe.py -q --summary
python scripts/probe.py --summary --output summary.json

6. Query Related Resources for a Specific Resource#

Query all resources related to a specific resource (e.g., ECS instance's security groups, vswitches, etc.):

bash

python scripts/probe.py --related-resource-id i-xxx --related-types security_group,vswitch --output related_items_ecs.json

This calls the GetProbeRelatedItems API, which returns all resources topologically related to the specified resource in the probe results. It does NOT create a new probe task. The attributes field of each related resource is stripped by default (same as probe results), which significantly reduces output size for resources like security groups that contain large Permissions arrays.

Two-step workflow (get a resource ID first, then query its related resources):

bash

# Step 1: Get resource list (use --skip-probe if already probed, -q for machine-readable output)
python scripts/probe.py --skip-probe -q --list-types ecs --regions cn-hangzhou

# Step 2: Pick any instance ID from Step 1 output, then query its related resources
python scripts/probe.py --related-resource-id i-bp1xxxxx --related-types security_group,vswitch --output related_items_ecs.json

Supported --related-types values (same as --list-types resource type keys):

• security_group — Security Group
• vswitch — VSwitch
• vpc — Virtual Private Cloud
• disk — Cloud Disk
• eni — Elastic Network Interface
• eip — Elastic IP Address
• ... (any resource type key listed in the supported resource types)

Leave --related-types empty to return all related resource types:

bash

python scripts/probe.py --related-resource-id i-xxx --output all_related.json

Output format (when using --related-resource-id):

json

{
  "mode": "related-items",
  "query_time": "2025-04-17 16:00:00",
  "resourceId": "i-bp1xxxxx",
  "serviceTypes": ["security_group", "vswitch"],
  "related_items": { ... }
}

How It Works#

The script implements resource probing through a two-phase flow using the BPStudio SDK:

Phase 1: Submit Probe Task#

Call ExecuteOperationSync API:

• ServiceType = probe
• Operation = CreateOneClickJob
• Attributes = {"regions": [...]}

The API returns an operation_id (task ID).

Phase 2: Poll for Results#

Call ExecuteOperationSync API:

• ServiceType = probe
• Operation = GetOneClickStatus
• Attributes = {"jobId": <jobId>}
• Poll every 10 seconds, timeout after 30 minutes, until the task completes
• When status == "SUCCESS", the arguments field contains the probed resource data (JSON)
• When status == "FAILURE", the message field contains the error information

Execution Modes#

• Default (full probe): CreateOneClickJob → poll GetOneClickStatus (every 10s, timeout 30min) until SUCCESS → GetProbeResourceSummary → GetProbeResult → GetLastProbeTime + GetProbeRegions (cache validation) → output
• --skip-probe: Skip CreateOneClickJob and polling, directly pull cached results via GetProbeResourceSummary → GetProbeResult → cache validation ( cache_info included in output)
• --related-resource-id: Skip full probe, directly call GetProbeRelatedItems with resourceId and optional serviceTypes filter → output. Requires a prior probe run; empty result if no probe data exists.

Cache Validation (All Modes)#

After fetching results, the script always calls GetLastProbeTime and GetProbeRegions to validate the cache state:

• cache_info (only in --skip-probe mode): Contains last_probe_minutes_ago (minutes since last probe, -1 if no valid job) and cached_regions (list of region IDs in cache)

• cache_warnings (all modes): Array of warning strings when:

  • No valid probe job exists ( lastTime == -1)
  • --regions includes regions not present in the cache → results for those regions will be empty

Output Description#

Console Output#

text

======================================================================
CADT Resource Probe (BPStudio)
======================================================================
  Probe time:   2025-04-17 15:30:00
======================================================================

Submitting probe task...
  Task ID: op-xxxxxxxxxxxxx
  Waiting for probe to complete...
  Current status: Running (elapsed 5s)
  Current status: SUCCESS (elapsed 15s)
  Probe complete!

======================================================================
Probe Results
======================================================================
<Resource list details>
======================================================================

JSON Output#

When using the --output flag, a full JSON file is written:

json

{
  "regions": ["cn-hangzhou", "cn-beijing"],
  "probe_time": "2025-04-17 15:30:00",
  "job_id": "12345",
  "progress": {"status": "SUCCESS"},
  "summary": { ... },
  "probe_result": { ... },
  "probe_result_filter": {
    "regions": "all",
    "serviceType": "all"
  }
}

--skip-probe Output (with cache_info)

When using --skip-probe, the output includes a cache_info field:

json

{
  "regions": ["cn-hangzhou"],
  "probe_time": "2025-04-17 16:00:00",
  "job_id": "",
  "progress": {},
  "cache_info": {
    "last_probe_minutes_ago": 45,
    "cached_regions": ["cn-hangzhou", "cn-beijing"]
  },
  "summary": { ... },
  "probe_result": { ... },
  "probe_result_filter": { ... }
}

--related-resource-id Output

When using --related-resource-id, the output structure is different from the full probe output:

json

{
  "mode": "related-items",
  "query_time": "2025-04-17 16:00:00",
  "resourceId": "i-bp1xxxxx",
  "serviceTypes": ["security_group", "vswitch"],
  "related_items": {
    "list": [
      { "resourceId": "sg-xxx", "resourceType": "security_group", ... },
      { "resourceId": "vsw-xxx", "resourceType": "vswitch", ... }
    ]
  }
}

Key fields:

• mode: Always "related-items" for this query type
• resourceId: The resource ID that was queried
• serviceTypes: The type filters applied (empty list means no filter)
• related_items: Raw response from GetProbeRelatedItems, containing a list of related resources

_relationships (Auto-extracted Relationship IDs)

Each resource in probe_result.list and related_items.list automatically includes a _relationships field (when applicable) that extracts cross-resource reference IDs from the nested attributes JSON. This field is added before attributes are stripped, so it is always available even without --keep-attributes.

Example — an ECS instance resource:

json

{
  "resourceId": "i-bp1xxxxx",
  "resourceType": "ecs",
  "_relationships": {
    "security_group_ids": ["sg-abc", "sg-def"],
    "vswitch_id": "vsw-123",
    "vpc_id": "vpc-456"
  }
}

Supported resource types and extracted relationships:

Resource TypeExtracted Fieldsecs``security_group_ids, vswitch_id, vpc_id``eni``security_group_ids, vswitch_id, vpc_id``rds, polardb, dds, kvstore, tair``vpc_id, vswitch_id``clb, slb, alb, nlb``vpc_id, vswitch_id (where applicable)nat, vpn``vpc_id, vswitch_id (where applicable)nas, hbase, clickhouse, lindorm, adb``vpc_id, vswitch_id``elasticsearch, kafka, emr, mse``vpc_id, vswitch_id (where applicable)disk``instance_id (references ECS)eip``instance_id (bound resource)

If a resource type is not in the table or no relationship fields are found, _relationships is omitted.

cache_warnings

When cache issues are detected, a cache_warnings array is added to the output (in all modes):

json

{
  "cache_warnings": [
    "Requested region(s) cn-shenzhen not in probe cache — results for these regions will be empty. Cached regions: cn-hangzhou, cn-beijing"
  ]
}

Observability#

User-Agent#

All Alibaba Cloud API calls made by this skill (SDK and CLI) include a custom User-Agent:

text

--user-agent AlibabaCloud-Agent-Skills/alibabacloud-cadt-probe/{session-id}

Rule: Every aliyun CLI command that calls a cloud API MUST include the --user-agent flag above. Local utility commands (e.g. configure, plugin, version) do not support this flag and should be excluded.

Session ID#

• Source: environment variable SKILL_SESSION_ID
• Fallback: if SKILL_SESSION_ID is not set, a random 32-character hex UUID is generated
• Scope: one session ID per invocation (created when the BPStudio client is initialized)

Exit Codes#

Exit CodeMeaning0Success1Failure (credentials not configured, probe task submission failed, API error, etc.)2Region cache miss — --regions includes regions not present in the probe cache. Results are still output, but the Agent should be aware that those regions returned empty data due to cache gaps, not because the regions have no resources.

Error Handling#

Credentials Not Configured#

text

Error: Unable to get credential. Please configure the default credential chain.

Probe Task Submission Failed#

Check:

• Whether credentials are correct
• Whether the account has BPStudio operation permissions (see references/ram-policies.md for the full permission list)
• Whether the Cloud Config service has been enabled

If the error is a permission error, follow the Permission Failure Handling process described in the RAM Policy section above.

Probe Task Timeout#

Default timeout is 30 minutes (1800 seconds). If it times out, the script outputs the current status and task ID, which can be used to query later.

Cache Warnings (stderr)#

Cache warnings are printed to stderr and included in the JSON output as cache_warnings:

• No valid probe job found in cache — results may be empty or stale
• Requested region(s) <regions> not in probe cache — results for these regions will be empty. Cached regions: <cached>

Supported Resource Types#

CADT probe supports 60+ Alibaba Cloud resource types. See resource_types.md for the full list.

File Structure#

text

alibabacloud-cadt-probe/
├── SKILL.md                    # This file
├── references/
│   ├── ram-policies.md         # Required RAM permissions
│   └── resource_types.md       # Supported resource types reference
└── scripts/
    └── probe.py                # Core probe script

Dependencies#

text

alibabacloud_bpstudio20210931==8.1.1
alibabacloud_credentials==1.0.5