Skip to content

chore(cli): define error actionability taxonomy#5775

Open
jgoux wants to merge 3 commits into
developfrom
julien/cli-1560-define-cli-error-actionability-taxonomy
Open

chore(cli): define error actionability taxonomy#5775
jgoux wants to merge 3 commits into
developfrom
julien/cli-1560-define-cli-error-actionability-taxonomy

Conversation

@jgoux

@jgoux jgoux commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Defines the CLI error actionability taxonomy for KPI reporting and adds a sanitized classifier contract for future cli_command_executed failure metadata.

This keeps the Q2 recovery and repeat definitions close to the telemetry catalog, while preserving the current behavior that command events only emit existing fields until the follow-up instrumentation issue wires classification into capture.

@jgoux jgoux marked this pull request as ready for review July 6, 2026 12:51
@jgoux jgoux requested a review from a team as a code owner July 6, 2026 12:51
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Supabase CLI preview

npx --yes https://pkg.pr.new/supabase/cli/supabase@99b1f00ac59ac0cca0b9c6b423288dd53148b436

Preview package for commit 99b1f00.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 511b9e3b61

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 786bba4590

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
@jgoux jgoux force-pushed the julien/cli-1560-define-cli-error-actionability-taxonomy branch from 786bba4 to b26bfe6 Compare July 6, 2026 13:27

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b26bfe6b63

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
@jgoux jgoux force-pushed the julien/cli-1560-define-cli-error-actionability-taxonomy branch from b26bfe6 to b8a178b Compare July 6, 2026 13:54

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b8a178ba4f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
@jgoux jgoux force-pushed the julien/cli-1560-define-cli-error-actionability-taxonomy branch from b8a178b to 8809e4c Compare July 6, 2026 14:49

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8809e4c704

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
@jgoux jgoux force-pushed the julien/cli-1560-define-cli-error-actionability-taxonomy branch from 8809e4c to d9f5099 Compare July 6, 2026 15:02

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d9f509914d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
@jgoux jgoux force-pushed the julien/cli-1560-define-cli-error-actionability-taxonomy branch from d9f5099 to bfe8271 Compare July 6, 2026 15:29

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bfe8271f0c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
@jgoux jgoux force-pushed the julien/cli-1560-define-cli-error-actionability-taxonomy branch from bfe8271 to 0aa23a6 Compare July 7, 2026 06:14

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0aa23a6145

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
@jgoux jgoux force-pushed the julien/cli-1560-define-cli-error-actionability-taxonomy branch from 0aa23a6 to 32045e5 Compare July 7, 2026 06:32

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 32045e5fb8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
@jgoux jgoux force-pushed the julien/cli-1560-define-cli-error-actionability-taxonomy branch from 32045e5 to d3c25fe Compare July 7, 2026 07:16

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d3c25fe0e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/shared/telemetry/error-actionability.ts Outdated
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
Comment thread apps/cli/src/shared/telemetry/error-actionability.ts
@jgoux jgoux force-pushed the julien/cli-1560-define-cli-error-actionability-taxonomy branch from d3c25fe to 99b1f00 Compare July 7, 2026 07:51

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 99b1f00ac5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

PlatformInputError: provideFlagsTemplate,
PlatformRouteNotFoundError: provideFlagsTemplate,
PlatformMethodSelectionError: provideFlagsTemplate,
LegacyInvalidOutputFormatError: invalidInputTemplate,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Map per-command env output rejections

Several legacy commands reject the globally valid -o env format with command-local tags, e.g. LegacyBranchesEnvNotSupportedError in apps/cli/src/legacy/commands/branches/list/list.handler.ts:52-55 and LegacyFunctionsEnvNotSupportedError in apps/cli/src/legacy/commands/functions/list/list.handler.ts:98-101. Since this table only maps LegacyInvalidOutputFormatError and the suffix fallback does not match EnvNotSupportedError, those deterministic flag errors are classified as unknown instead of invalid_input, so the output-format recovery KPI misses these failures.

Useful? React with 👍 / 👎.

Comment on lines +294 to +297
InvalidFunctionDeploySlugError: provideFlagsTemplate,
InvalidFunctionSlugError: provideFlagsTemplate,
ConflictingFunctionDownloadFlagsError: provideFlagsTemplate,
FunctionNotFoundError: provideFlagsTemplate,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include functions new validation tags

The function validation mappings here cover deploy/download/delete, but supabase functions new raises its own tags: invalid slugs fail as LegacyFunctionsNewInvalidSlugError and existing entrypoints fail as LegacyFunctionsNewFileExistsError with a concrete suggestion (apps/cli/src/legacy/commands/functions/new/new.handler.ts:196-203,228-234). Neither tag is in this table or matched by the suffix inference, so common functions new input/remediation failures fall through to unknown instead of the invalid-input bucket.

Useful? React with 👍 / 👎.

Comment on lines +290 to +293
LegacyStorageInvalidUrlError: provideFlagsTemplate,
LegacyStorageUrlParseError: provideFlagsTemplate,
LegacyStorageMissingFlagError: provideFlagsTemplate,
LegacyStorageMutuallyExclusiveFlagsError: provideFlagsTemplate,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Map network access validators as input

Network access commands validate user-supplied addresses before making API calls: network-bans remove raises LegacyNetworkBansInvalidIpError, and network-restrictions update raises LegacyNetworkRestrictionsInvalidCidrError / LegacyNetworkRestrictionsPrivateIpError (apps/cli/src/legacy/commands/network-bans/remove/remove.handler.ts:40-45; apps/cli/src/legacy/commands/network-restrictions/update/update.handler.ts:53-59). These tags are absent here and do not match the suffix inference, so malformed or private IP/CIDR arguments are recorded as unknown rather than user-actionable invalid input.

Useful? React with 👍 / 👎.

Comment on lines +283 to +289
LegacyExperimentalRequiredError: provideFlagsTemplate,
LegacyDbQueryMutuallyExclusiveFlagsError: provideFlagsTemplate,
LegacyDbPullTargetFlagsError: provideFlagsTemplate,
LegacyDbPullEngineConflictError: provideFlagsTemplate,
LegacyDeclarativeNotEnabledError: provideFlagsTemplate,
LegacyInvalidGenTypesDurationError: provideFlagsTemplate,
LegacyInvalidGenTypesDatabaseUrlError: provideFlagsTemplate,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Classify SSL enforcement flag validation

ssl-enforcement update fails locally with LegacySslEnforcementMutuallyExclusiveFlagsError when both enable/disable flags are set and LegacySslEnforcementNoEnableDisableFlagError when neither is set (apps/cli/src/legacy/commands/ssl-enforcement/update/update.handler.ts:48-53). Because these command-local flag errors are not mapped and have no useful suffix, deterministic bad flag combinations fall through to unknown instead of the invalid_input / provide_flags bucket.

Useful? React with 👍 / 👎.

Comment on lines +276 to +282
NoBranchNameError: provideFlagsTemplate,
BranchAlreadyExistsError: provideFlagsTemplate,
BranchNotFoundError: provideFlagsTemplate,
PlatformInputError: provideFlagsTemplate,
PlatformRouteNotFoundError: provideFlagsTemplate,
PlatformMethodSelectionError: provideFlagsTemplate,
LegacyInvalidOutputFormatError: invalidInputTemplate,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bucket project command argument failures

The legacy projects commands raise typed input failures before remote work: create can emit LegacyProjectsCreateMissingArgError or LegacyProjectsCreateNameEmptyError, and delete can emit LegacyProjectsDeleteRefRequiredError when a non-interactive ref is missing (apps/cli/src/legacy/commands/projects/create/create.handler.ts:40-46; apps/cli/src/legacy/commands/projects/projects.prompt.ts:55-58; apps/cli/src/legacy/commands/projects/delete/delete.handler.ts:56-61). None of these tags are in the taxonomy or suffix fallback, so project command argument mistakes are classified as unknown rather than invalid input.

Useful? React with 👍 / 👎.

Comment on lines +256 to +257
LegacyInvalidProjectRefError: invalidInputTemplate,
LegacyInvalidSecretPairError: invalidInputTemplate,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Classify secrets set local input failures

secrets set has local, user-remediable failures for missing values and --env-file problems: it raises LegacySecretsNoArgumentsError, LegacySecretsEnvFileOpenError, and LegacySecretsEnvFileParseError before uploading anything (apps/cli/src/legacy/commands/secrets/set/set.handler.ts:88-103,138-142). This table only maps malformed key/value pairs and config parsing, so those common secrets input/file failures fall through to unknown and lose the invalid-input recovery signal.

Useful? React with 👍 / 👎.

Comment on lines +200 to +202
LegacyDbConnectError: dbConnectionTemplate,
LegacyDbConfigConnectTempRoleError: dbConnectionTemplate,
LegacyDbConfigIpv6Error: dbConnectionTemplate,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Classify pooler login config mismatches

The DB config resolver has a distinct LegacyDbConfigPoolerLoginError for a configured pooler URL that does not match the linked project ref or active profile domain, and it can carry the same config-remediation suggestion shape as the adjacent DB config errors (apps/cli/src/legacy/shared/legacy-db-config.errors.ts:89-100). Because this table maps LegacyDbConfigConnectTempRoleError and LegacyDbConfigIpv6Error but not the pooler-login tag, those linked-database setup failures fall through to unknown instead of the db_connection / update-config bucket.

Useful? React with 👍 / 👎.

Comment on lines +283 to +289
LegacyExperimentalRequiredError: provideFlagsTemplate,
LegacyDbQueryMutuallyExclusiveFlagsError: provideFlagsTemplate,
LegacyDbPullTargetFlagsError: provideFlagsTemplate,
LegacyDbPullEngineConflictError: provideFlagsTemplate,
LegacyDeclarativeNotEnabledError: provideFlagsTemplate,
LegacyInvalidGenTypesDurationError: provideFlagsTemplate,
LegacyInvalidGenTypesDatabaseUrlError: provideFlagsTemplate,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Map postgres-config value parsing

postgres-config update rejects malformed --config entries locally with LegacyPostgresConfigInvalidConfigValueError before any API call when the value is not in key=value format (apps/cli/src/legacy/commands/postgres-config/update/update.handler.ts:33-37). That tag is absent from the invalid-input bucket and has no suffix fallback, so a deterministic flag-format error is reported as unknown rather than user-actionable input remediation.

Useful? React with 👍 / 👎.

Comment on lines +276 to +282
NoBranchNameError: provideFlagsTemplate,
BranchAlreadyExistsError: provideFlagsTemplate,
BranchNotFoundError: provideFlagsTemplate,
PlatformInputError: provideFlagsTemplate,
PlatformRouteNotFoundError: provideFlagsTemplate,
PlatformMethodSelectionError: provideFlagsTemplate,
LegacyInvalidOutputFormatError: invalidInputTemplate,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bucket invalid snippet IDs as input

snippets download validates the positional snippet ID as a UUID before issuing the request and raises LegacySnippetsInvalidIdError for malformed values (apps/cli/src/legacy/commands/snippets/download/download.handler.ts:68-72). Since this tag is not mapped and does not match any suffix inference, bad snippet IDs are classified as unknown instead of invalid_input, losing telemetry for a fully local argument error.

Useful? React with 👍 / 👎.

Comment on lines +253 to +255
LegacyProjectPausedError: {
...invalidConfigTemplate,
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Classify custom-domain CNAME setup failures

domains create performs a CNAME pre-check and raises LegacyDomainsCnameError when the user’s DNS record is missing or points at the wrong host, with the error text explaining the expected target (apps/cli/src/legacy/commands/domains/domains.cname.ts:72-83,87-91). This tag is neither mapped nor inferable by suffix, so a user-fixable DNS configuration problem is recorded as unknown instead of an invalid-config/update-config style bucket.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant