Huawei Cloud CCE Cost Optimization Advisor#

Overview#

Analyze CCE (Cloud Container Engine) cluster cost optimization opportunities. This skill performs read-only analysis and generates configuration suggestions — it does not directly modify HPA, autoscaler, node pools, or workload requests. All configuration changes require explicit user confirmation.

Analysis scope:

• 24-hour and 7-day node CPU/memory utilization trends
• Low-utilization node detection (below cluster average or below 30%)
• Oversized resource request detection (business workloads only)
• HPA and node autoscaler status review and recommendations
• Cost optimization report with execution plan

Architecture: Python SDK v3 → CCE API + AOM PromQL → Inventory + Metrics → Cost Analysis → Report

Security Constraints#

Dangerous Operation Confirmation Mechanism#

This skill enforces a strict read-only-by-default policy. All write operations require confirm=true.

Operations Requiring Confirmation

ToolOperation TypeRisk LevelDescriptionhuawei_configure_cce_hpaCreate/Update HPA🟠 HighCreates or replaces a HorizontalPodAutoscalerNode pool resize/scale-downScale🟠 HighReduces node pool capacity

Write operations without confirm=true return a preview only. The huawei_configure_cce_hpa tool returns a manifest preview and risk warning when called without confirm=true. Only after explicit user approval can it be called with confirm=true to apply the configuration.

Workflow

Step 1: Preview Operation — Call without confirm=true

bash

python3 scripts/huawei-cloud.py huawei_configure_cce_hpa \
  region=cn-north-4 \
  cluster_id=xxx \
  workload_name=my-deploy \
  namespace=default \
  min_replicas=1 \
  max_replicas=3 \
  target_cpu_utilization=60

Returns: HPA manifest preview, risk warning, confirmation hint

Step 2: Confirm Execution — Call with confirm=true after user approval

bash

python3 scripts/huawei-cloud.py huawei_configure_cce_hpa \
  region=cn-north-4 \
  cluster_id=xxx \
  workload_name=my-deploy \
  namespace=default \
  min_replicas=1 \
  max_replicas=3 \
  target_cpu_utilization=60 \
  confirm=true

Prohibited Actions#

• No automatic node pool scale-down — never delete nodes or shrink node pools automatically
• No workload request modification — never change CPU/memory requests directly
• No automatic HPA installation/update — never apply HPA without explicit user confirmation
• No autoscaler enable/disable — never toggle autoscaler without user approval

Allowed Actions#

• Read-only queries: nodes, node pools, pods, deployments, metrics, AOM PromQL
• Generate HPA YAML manifests, autoscaler parameter suggestions, and execution plans
• huawei_configure_cce_hpa without confirm=true returns preview only

Credential Security#

1. No persistent credential storage — AK/SK exists only during API calls
2. No credential leakage — never includes AK/SK in logs, responses, or errors
3. Environment variable preferred — HW_ACCESS_KEY / HW_SECRET_KEY / HW_REGION_NAME

---

Prerequisites#

Python Environment#

• Python 3.8+
• Install SDKs: pip install huaweicloudsdkcce huaweicloudsdkcore huaweicloudsdkces
• Optional for HPA operations: pip install kubernetes
• Optional for dashboard charts: pip install matplotlib numpy

Environment Variables (Recommended)#

bash

export HW_ACCESS_KEY="your-access-key-id"
export HW_SECRET_KEY="your-secret-access-key"
export HW_REGION_NAME="cn-north-4"

IAM Permission Policies#

Ensure the IAM user has the minimum required permissions:

PermissionDescriptioncce:cluster:listList clusterscce:cluster:getGet cluster detailscce:node:listList nodescce:node:getGet node detailscce:nodepool:listList node poolscce:nodepool:getGet node pool detailsaom:*:getRead AOM metrics and PromQL data

---

Core Commands#

Recommended: Combined Analysis#

ToolFunctionParametershuawei_analyze_cce_cost_optimizationOne-shot cost optimization analysis — inventory, 24h/7d node utilization, pod usage/request, HPA/autoscaler status, and report outputregion, cluster_id, exclude_namespaces, business_namespaces, short_hours, long_hours, top_n, output_dir

Prefer huawei_analyze_cce_cost_optimization for comprehensive analysis. Only use individual tools below for supplementing details, reviewing specific metrics, or manually generating HPA YAML.

Resource Inventory#

ToolFunctionParametershuawei_list_cce_clustersList all CCE clusters in regionregion``huawei_list_cce_nodesList cluster nodesregion, cluster_id``huawei_get_kubernetes_nodesGet Kubernetes node details (including allocatable resources)region, cluster_id``huawei_list_cce_nodepoolsList node pools with autoscaling inforegion, cluster_id``huawei_get_cce_podsGet pod list with labels, status, requestsregion, cluster_id``huawei_get_cce_deploymentsGet deployment listregion, cluster_id``huawei_list_cce_hpasList HPA configurations (excludes kube-system by default)region, cluster_id

Metrics Analysis#

ToolFunctionParametershuawei_get_cce_node_metrics_topNNode CPU/memory/disk utilization Top Nregion, cluster_id, top_n, hours``huawei_get_cce_node_metricsSingle node utilization time seriesregion, cluster_id, node_ip, hours``huawei_get_cce_pod_metrics_topNPod CPU/memory utilization Top N (supports custom PromQL)region, cluster_id, top_n, hours, cpu_query, memory_query``huawei_get_cce_pod_metricsSingle pod utilization time seriesregion, cluster_id, pod_name, namespace, hours``huawei_get_aom_metricsGeneric AOM PromQL queryregion, aom_instance_id, query, hours

Elasticity Policy#

ToolFunctionRisk LevelRequires Confirmationhuawei_generate_cce_hpa_manifestGenerate autoscaling/v2 HPA YAML (no cluster modification)🟢 LowNohuawei_configure_cce_hpaCreate or replace HPA in cluster🟠 HighYes ( confirm=true)

HPA configuration workflow:

1. Use huawei_generate_cce_hpa_manifest or huawei_configure_cce_hpa without confirm=true to generate a preview
2. Review the manifest with the user
3. Only after explicit user approval, call huawei_configure_cce_hpa with confirm=true

HPA recommendations must be based on request sizing. If requests are clearly oversized, first recommend calibrating requests, then configure HPA.

Dashboard#

ToolFunctionParametershuawei_generate_monitor_dashboardGenerate monitoring dashboard chart imagesregion, cluster_id, metrics_type, hours

---

Parameter Reference#

Common Parameters#

All tools accept these common parameters for authentication and region:

ParameterTypeRequiredDefaultDescriptionregionstringYes—Huawei Cloud region code (e.g., cn-north-4)cluster_idstringYes*—CCE cluster ID; not required for huawei_list_cce_clusters``akstringNoenv HW_ACCESS_KEYAccess Key ID; environment variable preferredskstringNoenv HW_SECRET_KEYSecret Access Key; environment variable preferredproject_idstringNoautoIAM project ID; auto-resolved from region if omitted

\* cluster_id is not required for huawei_list_cce_clusters (lists all clusters in region).

Combined Analysis Parameters ( huawei_analyze_cce_cost_optimization)#

ParameterTypeRequiredDefaultDescriptionregionstringYes—Huawei Cloud region codecluster_idstringYes—CCE cluster IDshort_hoursintNo24Short-window metrics duration in hourslong_hoursintNo168 (7d)Long-window metrics duration in hourstop_nintNo50Top N pods/nodes for oversized-request and utilization rankingexclude_namespacesstringNokube-systemComma-separated namespaces to exclude from analysisbusiness_namespacesstringNo—Comma-separated namespaces to treat as business workloads; if omitted, all non-excluded namespaces are analyzedoutput_dirstringNo—Directory to write summary JSON and report markdowninclude_rawboolNofalseInclude raw metrics data in output

HPA Parameters ( huawei_generate_cce_hpa_manifest / huawei_configure_cce_hpa)#

ParameterTypeRequiredDefaultDescriptionworkload_namestringYes—Target Deployment/StatefulSet namenamespacestringYes—Namespace of the target workloadmin_replicasintYes—Minimum replica count for HPAmax_replicasintYes—Maximum replica count for HPAworkload_typestringNodeploymentWorkload kind: deployment or statefulset``hpa_namestringNoautoHPA object name; defaults to <workload_name>-hpa``target_cpu_utilizationintNo60Target average CPU utilization percentagetarget_memory_utilizationintNo—Target average memory utilization percentage; omit to skip memory metricbehaviorobjectNo—HPA behavior policy (scaling rates, stabilization windows)confirmboolNofalsehuawei_configure_cce_hpa only: must be true to apply changes

Metrics Parameters ( huawei_get_cce_node_metrics_topN / huawei_get_cce_pod_metrics_topN)#

ParameterTypeRequiredDefaultDescriptiontop_nintNo10Number of top nodes/pods to returnhoursintNo1Metrics query time range in hourscpu_querystringNoautoCustom PromQL for CPU; defaults to built-in querymemory_querystringNoautoCustom PromQL for memory; defaults to built-in querynode_ipstringYes*—Required for huawei_get_cce_node_metrics (single node)pod_namestringYes*—Required for huawei_get_cce_pod_metrics (single pod)namespacestringYes*—Required for huawei_get_cce_pod_metrics (single pod)

\* Only required for single-entity metrics tools.

Dashboard Parameters ( huawei_generate_monitor_dashboard)#

ParameterTypeRequiredDefaultDescriptionhoursintNo1Monitoring data time range in hourstop_nintNo10Top N pods for dashboard rankingnamespacestringNo—Filter by namespacelabel_selectorstringNo—Filter by label (e.g., app=nginx)output_filestringNoautoOutput HTML file pathtitlestringNoautoDashboard title

---

Analysis Workflow#

See references/workflow.md for detailed analysis steps, thresholds, and decision logic.

Quick Summary#

1. Scope: Confirm region, cluster_id, namespace range, and exclusion rules (default: exclude kube-system)
2. Node utilization: Analyze 24h and 7d windows for CPU/memory usage per node and cluster average
3. Low-utilization detection: Flag nodes below cluster average by 20 percentage points or below 60% of cluster average; cluster average below 30% signals overall over-provisioning
4. Oversized requests: Compare business workload request vs actual p95 usage; mark as high (p95 < 33% of request), optimize (p95 < 50%), or observe (short-window only)
5. Elasticity review: Check node pool autoscaling and HPA status; generate recommendations
6. Output: Summary, utilization tables, oversized request list, HPA/autoscaler recommendations, risks, and verification steps

---

Risk Rules#

See references/risk-rules.md for complete safety boundaries.

Key constraints:

• Auto-execution limited to R1 read-only queries only
• No automatic scale-down, request modification, or HPA/autoscaler changes
• Must reference both 24h and 7d windows before recommending scale-down
• Cost optimization suggestions must include rollback strategy and verification metrics
• Data gaps (missing metrics, missing requests, invisible HPA) must be flagged in the report

---

Output Schema#

See references/output-schema.md for the complete JSON report structure.

All tools return JSON with:

• status / success: operation result
• data: analysis results, metrics, or configuration preview
• message: human-readable description
• warning: risk warning for write operations (preview mode only)
• files: paths to generated summary JSON and report markdown

---

Supported Regions#

Region CodeRegion Namecn-north-4North China-Beijing 4cn-north-1North China-Beijing 1cn-east-3East China-Shanghai 1cn-south-1South China-Guangzhouap-southeast-1Asia-Pacific-Hong Kongap-southeast-2Asia Pacific-Bangkokap-southeast-3Asia Pacific-Singapore

---

Best Practices#

1. Run the combined analysis first — use huawei_analyze_cce_cost_optimization for a complete picture before drilling into individual tools; avoid piecemeal queries that miss cross-resource dependencies.
2. Always check both time windows — rely on 7-day data for stable optimization decisions; use 24-hour data only for short-term fluctuation observation, never as the sole basis for scale-down recommendations.
3. Exclude kube-system by default — system workloads have fixed sizing requirements; analyzing them produces misleading oversized-request signals and wastes analysis capacity.
4. Calibrate requests before configuring HPA — HPA scales based on request percentages; if requests are oversized, HPA will trigger premature scaling. Fix requests first, then set HPA targets.
5. Use environment variables for credentials — prefer HW_ACCESS_KEY / HW_SECRET_KEY over passing AK/SK as parameters to avoid credential leakage in command history and logs.
6. Review HPA preview before confirming — always call huawei_configure_cce_hpa without confirm=true first; inspect the manifest YAML and risk warning with the user before applying.
7. Include rollback strategy in every recommendation — cost optimization changes can impact availability; every suggestion must specify how to revert and how to verify the change was safe.
8. Flag data gaps explicitly — if metrics are missing, requests are absent, or HPA status is invisible, report these as data gaps; do not infer optimization decisions from incomplete data.
9. Set top_n appropriately — use top_n=50 for large clusters (100+ pods) to capture all significant outliers; reduce to top_n=10 for focused analysis of specific namespaces.
10. Save outputs to a persistent directory — use output_dir to write the summary JSON and report markdown to a known location; this enables later review and comparison across multiple analysis runs.

---

Common Pitfalls#

PitfallSymptomQuick FixMissing AK/SK credentialsAll tools return "success": false with credential errorSet HW_ACCESS_KEY and HW_SECRET_KEY environment variables before runningWrong cluster IDEmpty or error results from cluster-specific toolsRun huawei_list_cce_clusters first to confirm the correct cluster_id for your regionAnalyzing kube-system workloadsFalse oversized-request alerts on system DaemonSetsSet exclude_namespaces=kube-system (default) or add other system namespacesSingle-window scale-down decisionNode marked low-utilization in 24h only but stable in 7dAlways require both short_hours=24 and long_hours=168 before recommending scale-downHPA on oversized requestsHPA triggers scaling at low actual usage because requests are inflatedFirst reduce CPU/memory requests to realistic values, then configure HPA with target_cpu_utilization=60Missing AOM metricsEmpty utilization data, data_gaps flagged in reportVerify IAM has aom:*:get permission and AOM is enabled on the clusterApplying HPA without previewhuawei_configure_cce_hpa called with confirm=true without reviewAlways call without confirm=true first, review manifest, then re-run with confirm=truekubernetes SDK not installedHPA tools fail with "Kubernetes SDK not installed"Install with pip install kubernetes before using HPA listing or configuration toolsLarge cluster with small top_nOversized-request pods missing from reportIncrease top_n to 50 or higher for clusters with 100+ business podsNo output directory specifiedReport files written to temporary location, may be lostSet output_dir to a persistent path like ./cost-reports

---

Output Format#

All tools return JSON with status, success, data, message, warning, and iles fields. See references/output-schema.md for the complete report structure.

Verification#

See Verification Method for step-by-step verification.

Cross-Skill References#

SkillWhen to Usehuawei-cloud-cce-cluster-managementCreate/delete/hibernate clusters, manage node pools, manage addons, cordon/uncordon/drain nodes, create/delete individual nodes

---

Reference Documents#

DocumentPathDescriptionWorkflowreferences/workflow.mdDetailed analysis workflow, thresholds, and decision logicRisk Rulesreferences/risk-rules.mdSafety boundaries, prohibited actions, and confirmation requirementsOutput Schemareferences/output-schema.mdCost optimization report JSON structure

---

Notes#

• Ensure AK/SK has correct IAM permissions (CCE read + AOM read)
• Default analysis excludes kube-system namespace
• HPA recommendations require request sizing to be reasonable first
• Node scale-down suggestions require both 24h and 7d data confirmation
• Cost optimization reports must include rollback strategy
• Data gaps must be explicitly flagged