Skip to content

Add queue auto-discovery support for BullMQ meta keys#134

Open
Copilot wants to merge 5 commits into
masterfrom
copilot/feat-add-support-queue-auto-discovery
Open

Add queue auto-discovery support for BullMQ meta keys#134
Copilot wants to merge 5 commits into
masterfrom
copilot/feat-add-support-queue-auto-discovery

Conversation

Copilot AI commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Queue auto-discovery currently relies on :id keys, which causes modern BullMQ queues to be missed when only :meta exists (e.g., worker/queue initialized before any jobs). This change extends discovery to include :meta keys while preserving existing queue parsing and typing behavior.

  • Discovery behavior

    • Scan Redis for both queue key patterns:
      • *:*:id (TYPE string)
      • *:*:meta (TYPE hash)
    • Normalize discovered :meta keys to canonical :id queue identifiers so downstream logic remains unchanged.
    • Deduplicate queues found by both key types.
  • Queue key parsing hardening

    • Replace regex-based queue key parsing with delimiter-based parsing (lastIndexOf) for id|meta suffixes.
    • Keep the same extracted queue identity (prefix, name) semantics used by discovery and warning paths.
  • Focused coverage

    • Add targeted tests for:
      • discovering a queue when only :meta exists
      • avoiding duplicate queue entries when both :id and :meta exist
// discovery now checks both key families
SCAN ... MATCH "*:*:id"   TYPE string
SCAN ... MATCH "*:*:meta" TYPE hash

// normalize meta key to canonical queue key
bull:emails:meta -> bull:emails:id

Copilot AI changed the title [WIP] Add support in queue auto discovery for meta keys Add queue auto-discovery support for BullMQ meta keys Jun 15, 2026
Copilot AI requested a review from manast June 15, 2026 21:39
@manast manast requested a review from Copilot June 15, 2026 21:50

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends Bull/BullMQ queue auto-discovery so BullMQ queues are found even when only :meta exists (e.g., before any jobs create :id keys), while keeping downstream queue handling based on canonical :id identifiers.

Changes:

  • Add :meta (TYPE hash) scanning alongside existing :id (TYPE string) scanning, normalizing :meta:id and deduplicating results.
  • Replace regex-based queue key parsing with delimiter-based parsing (lastIndexOf) supporting both id and meta suffixes.
  • Add tests covering meta-only discovery and de-duplication when both key types exist.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
lib/queue-factory.ts Adds meta-key scanning + key parsing hardening, and dedupes discovered queues.
tests/queue-factory.spec.js Adds targeted tests for BullMQ meta-key discovery and deduplication.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/queue-factory.ts Outdated
Comment thread tests/queue-factory.spec.js

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@manast manast marked this pull request as ready for review June 17, 2026 06:47
@manast manast requested a review from roggervalf June 17, 2026 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants