Skip to content

Auto-assign Copilot to next- issues generated from weekly run, and add fix-pyright skill#46726

Open
JennyPng wants to merge 19 commits intoAzure:mainfrom
JennyPng:automate-fixing
Open

Auto-assign Copilot to next- issues generated from weekly run, and add fix-pyright skill#46726
JennyPng wants to merge 19 commits intoAzure:mainfrom
JennyPng:automate-fixing

Conversation

@JennyPng
Copy link
Copy Markdown
Member

@JennyPng JennyPng commented May 5, 2026

core:

  • experiment: auto assign gh copilot to create a PR for any detected next-* issues generated from the weekly pipeline
  • checks for existing open PRs tied to the issue, only assigns copilot if none exist
  • reassigns Copilot to an existing issue if the version of the check mentioned was updated
  • assigns copilot by querying for its bot node ID dynamically - did this because I think it could change if the agent session ever gets restarted ....? it's just unclear if this ID may change or not
    • using a GraphQL endpoint because it's better documented for this purpose than assigning through Rest (related thread)
  • expects gh copilot to leverage fix-* skills for all the next- checks

misc.

  • adds a fix-pyright skill which matches existing skills for fixing pylint, sphinx, black, mypy
  • removes a mention of ruff that was previously missed, since we removed it entirely from the repo

tests:

@JennyPng JennyPng marked this pull request as ready for review May 5, 2026 22:51
@JennyPng JennyPng requested a review from scbedd as a code owner May 5, 2026 22:51
Copilot AI review requested due to automatic review settings May 5, 2026 22:51
@JennyPng JennyPng requested review from a team, benbp, l0lawrence and mccoyp as code owners May 5, 2026 22:51
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the eng/tools/azure-sdk-tools vnext issue automation to optionally auto-assign the Copilot coding agent for eligible next-* issues (with duplicate PR detection and version-bump retriggering), and adds a new fix-pyright skill to align with existing fix-* skills.

Changes:

  • Add Copilot auto-fix helpers to vnext issue creation/update flow (labels, instruction block upsert, duplicate PR detection, GraphQL assignment, and optional force-reassign on version change).
  • Add unit tests covering eligibility, duplicate PR detection, instruction generation, label reconciliation, and assignment flows.
  • Add fix-pyright skill documentation and update engineering docs (including removing the outdated Ruff row).

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
eng/tools/azure-sdk-tools/tests/test_vnext_auto_fix.py Adds tests for Copilot auto-fix helpers and integration behavior.
eng/tools/azure-sdk-tools/gh_tools/vnext_issue_creator.py Implements Copilot auto-fix logic (instructions/labels/assignment) and updates some type signatures and mgmt labeling heuristic.
doc/eng_sys_checks.md Documents the Copilot auto-fix behavior for weekly next-* checks.
doc/analyze_check_versions.md Removes the Ruff row from the checker version table.
.github/skills/fix-pyright/SKILL.md Introduces a new skill guide for fixing pyright issues, consistent with other fix-* skills.

Comment thread eng/tools/azure-sdk-tools/gh_tools/vnext_issue_creator.py Outdated
Comment on lines +156 to +158
end_idx = body.find(COPILOT_AUTOFIX_END)
if start_idx != -1 and end_idx != -1:
return body[:start_idx].rstrip() + instructions
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i dont think we ever expect human authored content in the issue?

Comment on lines +107 to +116
try:
open_prs = repo.get_pulls(state="open", sort="created", direction="desc")
for pr in open_prs:
body = pr.body or ""
title = pr.title or ""
search_text = f"{title} {body}".lower()

# 1. PR explicitly references the issue number
issue_ref = f"#{issue_number}"
has_issue_ref = issue_ref in title or issue_ref in body
Comment thread eng/tools/azure-sdk-tools/gh_tools/vnext_issue_creator.py Outdated
Comment thread eng/tools/azure-sdk-tools/gh_tools/vnext_issue_creator.py Outdated
Comment thread doc/eng_sys_checks.md Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
**Command:**
```powershell
cd <package-path>
azpysdk --isolate pyright .
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes eng/tools/azure-sdk-tools is installed. We probably point to it in a skill that references this? if not, we need to repeat ourselves I think.


**Command:**
```powershell
cd <package-path>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
cd <package-path>
cd <package-path>
# install dev_requirements.txt

**If virtual environment is missing:**
Ask: "Do you have an existing virtual environment path, or should I create 'env'?"

### Step 1: CRITICAL - Activate Virtual Environment FIRST
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • analyze are run on python 3.10
  • We should mention this when deciding to re-use or active a venv.

**IMMEDIATELY activate the virtual environment before ANY other command:**

```powershell
# Activate the provided virtual environment (e.g., env, venv)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Real talk. agents are quite good at activating venvs. As long as you tell them activate a venv before all commands it should take care of it crossplat.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just make certain it knows what version of python to prep.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants