Skip to content

Add worker-name filter to airflow edge list-workers#70095

Merged
dheerajturaga merged 2 commits into
apache:mainfrom
shubhamraj-git:create-subcommand-for-edge-filter
Jul 20, 2026
Merged

Add worker-name filter to airflow edge list-workers#70095
dheerajturaga merged 2 commits into
apache:mainfrom
shubhamraj-git:create-subcommand-for-edge-filter

Conversation

@shubhamraj-git

@shubhamraj-git shubhamraj-git commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

In clusters with many edge workers, list-workers could only be narrowed by --state. This adds an optional --worker-name-pattern glob (e.g. 'prod-*', '*gpu*') that filters workers by name DB-side, composable with --state. This brings the CLI to parity with the edge UI/REST route, which already supports worker_name_pattern.

Usage:
airflow edge list-workers --worker-name-pattern 'bree-*'

image
Was generative AI tooling used to co-author this PR?
  • Yes (Opus 4.8)

@boring-cyborg boring-cyborg Bot added area:providers kind:documentation provider:edge Edge Executor / Worker (AIP-69) / edge3 labels Jul 19, 2026
@shubhamraj-git shubhamraj-git added the ready for maintainer review Set after triaging when all criteria pass. label Jul 19, 2026
@shubhamraj-git shubhamraj-git added this to the Airflow 3.3.1 milestone Jul 19, 2026
@dheerajturaga

dheerajturaga commented Jul 20, 2026

Copy link
Copy Markdown
Member

@shubhamraj-git, looks like there are conflicts. Can you resolve them?

What are your thoughts on extending the --edge-hostname to accept a pattern instead. Given we already have the argument we may be able to use the same variable rather than adding a new argument

@dheerajturaga dheerajturaga left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually on second pass, the current approach is better. Could we rename this option from --name-pattern to --worker-name-pattern?

The command output and model field both use worker_name, and the edge API/UI already expose worker_name_pattern, so --worker-name-pattern makes the target field clearer. I'd avoid extending --edge-hostname for this, since that option is used as an exact worker identity for mutation commands.


Drafted-by: Codex (GPT-5); reviewed by @dheerajturaga before posting

@shubhamraj-git

Copy link
Copy Markdown
Contributor Author

Actually on second pass, the current approach is better. Could we rename this option from --name-pattern to --worker-name-pattern?

The command output and model field both use worker_name, and the edge API/UI already expose worker_name_pattern, so --worker-name-pattern makes the target field clearer. I'd avoid extending --edge-hostname for this, since that option is used as an exact worker identity for mutation commands.

Drafted-by: Codex (GPT-5); reviewed by @dheerajturaga before posting

I gave it a thought before raising this PR,
but

airflow edge list-workers --worker-name-pattern PATTERN

sounds bit redundant on the worker side, so thought of going with just name-pattern since here name self explains it is for worker, let me know what do you think?

@shubhamraj-git
shubhamraj-git force-pushed the create-subcommand-for-edge-filter branch from 0d78a59 to 0bea07c Compare July 20, 2026 18:47
@dheerajturaga

Copy link
Copy Markdown
Member

Actually on second pass, the current approach is better. Could we rename this option from --name-pattern to --worker-name-pattern?
The command output and model field both use worker_name, and the edge API/UI already expose worker_name_pattern, so --worker-name-pattern makes the target field clearer. I'd avoid extending --edge-hostname for this, since that option is used as an exact worker identity for mutation commands.
Drafted-by: Codex (GPT-5); reviewed by @dheerajturaga before posting

I gave it a thought before raising this PR, but

airflow edge list-workers --worker-name-pattern PATTERN

sounds bit redundant on the worker side, so thought of going with just name-pattern since here name self explains it is for worker, let me know what do you think?

just incase we want to use the same option for other edge CLIs it would make it clear

Match the existing worker_name_pattern naming already used by the edge
REST/UI routes for consistency.

@dheerajturaga dheerajturaga left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Works as expected. Thanks for making adjustments as needed!

@dheerajturaga
dheerajturaga merged commit 154399c into apache:main Jul 20, 2026
79 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers kind:documentation provider:edge Edge Executor / Worker (AIP-69) / edge3 ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants