Skip to content

PR: Scaffold automated GitHub issue labeling workflow and contract#77

Open
ashleyshaw wants to merge 6 commits into
developfrom
claude/add-labeling-workflow-scaffolds-011CV45vmxaoRCgss25C3v4M
Open

PR: Scaffold automated GitHub issue labeling workflow and contract#77
ashleyshaw wants to merge 6 commits into
developfrom
claude/add-labeling-workflow-scaffolds-011CV45vmxaoRCgss25C3v4M

Conversation

@ashleyshaw
Copy link
Copy Markdown
Member

@ashleyshaw ashleyshaw commented Nov 12, 2025


name: "Pull Request"
about: "General changes, refactors, and maintenance"
title: "PR: Scaffold automated GitHub issue labeling workflow and contract enforcement"
labels: ["status:needs-review"]

General Pull Request

This repository enforces changelog, release, and label automation for all PRs and issues.
See the organisation-wide Automation Governance & Release Strategy for contributor rules.

Add comprehensive scaffolding for the labeling automation workflow system:

  • Update labeling.yml workflow to use a dedicated agent script with dry-run and Markdown report generation.
  • Add report-writer.js for generating rich labeling reports, including telemetry and audit placeholders.
  • Create TEMPLATE_CONTRACT.md outlining the required structure and policy for label/type templates.
  • Refactor manage-labels.sh with robust CLI argument parsing and new --prune safety mechanism.
  • Add test-template-labels.js for contract assurance, validating that all template labels match the canonical set.
  • Simplify SAVED_REPLIES/README.md and strengthen linker guidance for labels on the develop branch.

These changes lay the foundation for resilient, testable automation of label management, enforce validation of label contracts, and improve workflow observability. All code aligns with LightSpeed coding standards, accessibility best practices, and UK English conventions.

Ref: Labeling workflow audit and automation governance requirements.

Linked issues

Closes #

Changelog

Added

  • labeling.yml workflow now delegates to an agent script supporting dry-runs and reporting.
  • report-writer.js: Generates Markdown audits for labeling workflow results and planned actions.
  • TEMPLATE_CONTRACT.md: Documents canonical structure, rules, and validation policy for label/type templates.
  • test-template-labels.js: Ensures template label sets conform to contract.
  • Enhanced PR labeling automation and documentation guidance.

Changed

  • manage-labels.sh now supports argument parsing and a --prune flag to prevent accidental mass deletions.
  • SAVED_REPLIES/README.md refactored for clarity and aligned with develop branch label templates.

Fixed

  • N/A

Removed

  • N/A

Risk Assessment

Risk Level: Low

Potential Impact:

  • Errors in workflow/label automation could cause incomplete or overly broad labeling, affecting issue and PR triage.
  • The new scripts are controlled by test coverage and isolated from existing production labeling.

Mitigation Steps:

  • All new scripts have contract tests (test-template-labels.js) and local dry-run capability.
  • Documentation scaffolds and inline comments clarify usage and rollout.
  • Workflow changes are staged for review and deployed on a feature branch.

How to Test

Prerequisites

  • NodeJS environment for running test/template scripts
  • Ability to run workflows in dry-run mode (DRY_RUN=true)
  • Repo fork/feature branch access

Test Steps

  1. Validate label contract: Run node test-template-labels.js. All template/label sets should pass validation with no errors.
  2. Dry-run workflow: Execute labeling.yml workflow with dry-run mode enabled. Confirm report Markdown is generated and no labels are modified.
  3. Prune safety: Run manage-labels.sh --prune in a non-production/test repo and confirm only unused labels are flagged/deleted with confirmation.
  4. Review documentation: Inspect TEMPLATE_CONTRACT.md and see if guidance is clear and complete.
  5. Bootstrap real run: On staging/test, disable dry-run and run labeling workflow end to end. Confirm only labels meeting contract are applied/modified.

Expected Results

  • All scripts/tests pass, and the generated reports are accurate and human-readable.
  • No production labels or issues are modified during dry-runs; actual label changes are as expected when enabled.
  • Template contract documentation provides unambiguous guidance.

Edge Cases to Verify

  • Mismatched or missing canonical labels in template throw contract test/test errors
  • Workflow gracefully handles empty or malformed templates
  • manage-labels.sh aborts on unexpected argument or unsafe use of --prune
  • All output and reports are accessible and markdown-formatted
  • Workflows are re-runnable and idempotent

Checklist (Global DoD / PR)

  • All AC met and demonstrated
  • Tests added/updated (unit/E2E as appropriate)
  • A11y considerations addressed where relevant
  • Docs/readme/changelog updated (if user-facing)
  • Security/perf impact reviewed where relevant
  • Code/design reviews approved
  • CI green; linked issues closed; release notes prepared (if shipping)
  • Risk assessment completed above
  • Testing instructions provided above

References


Add comprehensive scaffolding for the labeling automation workflow system:

- Update labeling.yml workflow to use dedicated agent script with dry-run and report generation
- Add report-writer.js for generating Markdown labeling reports with telemetry placeholders
- Create TEMPLATE_CONTRACT.md documenting label/type contract requirements
- Refactor manage-labels.sh with new CLI argument parsing and --prune flag support
- Add test-template-labels.js contract test to validate template labels against canonical set
- Simplify SAVED_REPLIES/README.md with focus on develop branch label placeholders

These scaffolds provide the foundation for automated label management, template validation,
and reporting whilst maintaining separation between draft development and production systems.
All changes adhere to LightSpeed coding standards and UK English conventions.

Ref: Labeling workflow audit and automation governance requirements
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Frontmatter validation failed

Please check the workflow logs for details and ensure all frontmatter follows the schema at schemas/frontmatter/frontmatter.schema.json.

See Frontmatter Documentation for guidance.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Nov 12, 2025

Warning

Rate limit exceeded

@ashleyshaw has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 1 minutes and 14 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 39ecde7 and 16a9efc.

📒 Files selected for processing (6)
  • .github/SAVED_REPLIES/README.md (1 hunks)
  • .github/agents/includes/report-writer.js (1 hunks)
  • .github/workflows/labeling.yml (1 hunks)
  • docs/label-automation/TEMPLATE_CONTRACT.md (1 hunks)
  • scripts/maintenance/manage-labels.sh (1 hunks)
  • tests/contracts/test-template-labels.js (1 hunks)

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Note

.coderabbit.yml has unrecognized properties

CodeRabbit is using all valid settings from your configuration. Unrecognized properties (listed below) have been ignored and may indicate typos or deprecated fields that can be removed.

⚠️ Parsing warnings (1)
Validation error: Unrecognized key(s) in object: 'auto_labels', 'auto_assign', 'auto_review'
⚙️ Configuration instructions
  • Please see the configuration documentation for more information.
  • You can also validate your configuration using the online YAML validator.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch claude/add-labeling-workflow-scaffolds-011CV45vmxaoRCgss25C3v4M

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request establishes foundational infrastructure for automated GitHub issue and pull request labeling. The changes introduce scaffolding for label management workflows, template validation contracts, and reporting mechanisms while maintaining separation between development and production systems.

Key changes:

  • New contract test to validate issue template labels against canonical label definitions
  • Simplified label management script with modern CLI argument parsing
  • Labeling workflow refactored to use dedicated agent with dry-run and report generation capabilities

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/contracts/test-template-labels.js Adds contract test validating issue template labels exist in canonical labels.yml
scripts/maintenance/manage-labels.sh Refactors to stub implementation with CLI arg parsing for --token, --org, --repo, --dry-run, --prune
docs/label-automation/TEMPLATE_CONTRACT.md Documents contract requirements between templates and canonical label/type definitions
.github/workflows/labeling.yml Simplifies workflow to call labeling agent with report generation and optional commit
.github/agents/includes/report-writer.js Adds report generator with placeholder telemetry for labeling run metrics
.github/SAVED_REPLIES/README.md Simplifies to focus on label placeholder usage in develop branch
Comments suppressed due to low confidence (1)

scripts/maintenance/manage-labels.sh:1

  • The previous implementation checked DRY_RUN environment variable but the new version uses $DRY variable. If existing workflows or documentation reference DRY_RUN=true, this breaking change will cause the flag to be ignored.
#!/usr/bin/env bash

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread scripts/maintenance/manage-labels.sh Outdated
Comment on lines +24 to +29
# TODO: read canonical labels from .github/automation/labels.yml
# Apply: create/update labels
# If $PRUNE, remove labels not in canonical set (respect deprecations list).
# Honour $DRY to only print planned changes.

fetch_canonical_labels
get_repository_list
for repo in "${REPOS[@]}"; do
sync_repository_labels "$repo"
done
echo "Done."
# shellcheck disable=SC2317,SC2329
exit 0
echo "[INFO] Manage labels (org=$ORG repo=$REPO dry=$DRY prune=$PRUNE)"
Copy link

Copilot AI Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script has been reduced to a stub with only argument parsing and a placeholder echo statement. All label synchronization logic (fetching canonical labels, listing repositories, syncing labels) has been removed. Consider documenting that this is intentionally a stub implementation in the TODO comments, or implement the basic functionality to avoid merging non-functional code.

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/labeling.yml Outdated
Comment thread .github/workflows/labeling.yml Outdated
Comment thread .github/workflows/labeling.yml Outdated
Comment thread .github/SAVED_REPLIES/README.md
Comment thread scripts/maintenance/manage-labels.sh Outdated
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Ash Shaw <ashley@lightspeedwp.agency>
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Frontmatter validation failed

Please check the workflow logs for details and ensure all frontmatter follows the schema at schemas/frontmatter/frontmatter.schema.json.

See Frontmatter Documentation for guidance.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Ash Shaw <ashley@lightspeedwp.agency>
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Frontmatter validation failed

Please check the workflow logs for details and ensure all frontmatter follows the schema at schemas/frontmatter/frontmatter.schema.json.

See Frontmatter Documentation for guidance.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Ash Shaw <ashley@lightspeedwp.agency>
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Frontmatter validation failed

Please check the workflow logs for details and ensure all frontmatter follows the schema at schemas/frontmatter/frontmatter.schema.json.

See Frontmatter Documentation for guidance.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Ash Shaw <ashley@lightspeedwp.agency>
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Frontmatter validation failed

Please check the workflow logs for details and ensure all frontmatter follows the schema at schemas/frontmatter/frontmatter.schema.json.

See Frontmatter Documentation for guidance.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Ash Shaw <ashley@lightspeedwp.agency>
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Frontmatter validation failed

Please check the workflow logs for details and ensure all frontmatter follows the schema at schemas/frontmatter/frontmatter.schema.json.

See Frontmatter Documentation for guidance.

@ashleyshaw ashleyshaw changed the title Set up automated GitHub issue labeling workflow PR: Scaffold automated GitHub issue labeling workflow and contract Nov 13, 2025
@ashleyshaw ashleyshaw closed this May 19, 2026
@ashleyshaw ashleyshaw reopened this May 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants