Skip to content

[BUG] Parameter descriptions returned from extension_cli_generate hallucinated #2047

@silkfire

Description

@silkfire

Describe the bug

When using the extension_cli_generate tool, instead of sourcing the parameter descriptions from the Azure CLI source code metadata or help section, it seems the MCP server creates its own descriptions, which can lead to subtle inaccuracies.

E.g. I tested the extension_cli_generate tool by asking the AI to generate a query to move resources from one resource group to another, and the response's placeholderSet describes the --ids parameter as:

Comma-separated list of resource IDs of the resources to be moved.

This is incorrect. The --ids parameter for az resource move accepts space-separated resource IDs, not comma-separated.

Expected behavior

The placeholder description for $resourceIds should say Space-separated list of resource IDs of the resources to be moved, consistent with the Azure CLI source and documentation.

Actual behavior

The generated placeholderSet entry says:

{
  "name": "$resourceIds",
  "desc": "Comma-separated list of resource IDs of the resources to be moved.",
  "type": "string",
  "optional": false
}

Meanwhile, the generated example in the same response correctly shows space-separated IDs:

az resource move --destination-group my-new-rg --ids "/subscriptions/<sub-id>/resourceGroups/my-old-rg/providers/Microsoft.Storage/storageAccounts/mystorage" "/subscriptions/<sub-id>/resourceGroups/my-old-rg/providers/Microsoft.Compute/virtualMachines/myvm"

And the Azure CLI source (_help.py) confirms space-separated usage:

az resource move --destination-group ResourceGroup --destination-subscription-id SubscriptionId --ids "ResourceId1" "ResourceId2" "ResourceId3"

Reproduction Steps

  1. Call the extension_cli_generate tool with:
    1. intent: "Move resources from one resource group to another resource group"
    2. cli-type: "az"
  2. Observe the placeholderSet in the response — it incorrectly describes --ids as comma-separated.

Additional context:

The contradiction is within the tool's own response: the example uses space-separated IDs, but the placeholder description says "comma-separated". A user following the description rather than the example would produce a malformed command.

Environment

Windows 11
VS Code 1.111.0
Claude Code extension 2.1.76

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Status

Untriaged

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions