Add working agentic issue triage workflows#5527
Merged
andyleejordan merged 1 commit intoJun 18, 2026
mainPowerShell/vscode-powershell:mainfrom
andyleejordan/fix-issue-triage-workflowPowerShell/vscode-powershell:andyleejordan/fix-issue-triage-workflowCopy head branch name to clipboard
Merged
Add working agentic issue triage workflows#5527andyleejordan merged 1 commit intomainPowerShell/vscode-powershell:mainfrom andyleejordan/fix-issue-triage-workflowPowerShell/vscode-powershell:andyleejordan/fix-issue-triage-workflowCopy head branch name to clipboard
andyleejordan merged 1 commit into
mainPowerShell/vscode-powershell:mainfrom
andyleejordan/fix-issue-triage-workflowPowerShell/vscode-powershell:andyleejordan/fix-issue-triage-workflowCopy head branch name to clipboard
Conversation
The workflows added in #5409 were broken and never customized. They couldn't see spam from non-collaborators (wrong `roles`/`min-integrity`), couldn't close issues (missing `close-issue` safe-output), and failed auth (under-scoped `COPILOT_GITHUB_TOKEN`). Fixed and expanded into three workflows: - `issue-triage.md`: Incoming issue triage on open/reopen. Closes spam and duplicates as "not planned", requests author feedback when reproduction is missing, or labels real issues with `Needs: Triage` plus area/platform/type labels and a maintainer hand-off note (includes reproducibility assessment and Copilot-fix suitability). - `issue-comment-handler.md`: Smart comment processor on `issue_comment`. When users confirm a fix is resolved, thanks them and closes the issue (less manual work for maintainers). When users respond to `Needs: Author Feedback` with substance, moves back to `Needs: Triage`. Politely educates on non-actionable bumps. Auto-labels based on keywords (Area-*, OS-*, Bug-*). Summarizes threads over 10 comments. Flags duplicate mentions. - `stale-closer.md`: Weekly intelligent backlog cleanup. Closes up to 10 issues/week using AI judgment (no reproduction after 90+ days, duplicates of closed issues, external blockers declined upstream, features out of scope). Skips anything with maintainer activity in last 6 months. Conservative: when in doubt, keeps open. Deleted `close-stale-issues.yml` (Issue Housekeeping) — it used `actions/stale` with time-based rules that didn't work well. Replaced by `stale-closer.md` using judgment. Switched all three to `permissions: copilot-requests: write` instead of PAT for centralized org billing and no secret management. Upgraded gh-aw from v0.49.5 to v0.79.8. Deleted auto-generated `agentics-maintenance.yml` (none of our workflows use `expires` so we don't need it). Expected workflow: new issues get triaged/closed within minutes, users confirming fixes auto-close their issues, and maintainers filter to `Needs: Triage` to see only real, actionable, pre-labeled issues. Drafted by Copilot (Claude Opus 4.8). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR replaces the broken, uncustomized agentic issue triage workflow (from #5409) and the time-based actions/stale issue housekeeping workflow with three focused gh-aw agentic workflows tailored to the PowerShell/vscode-powershell repository. The new workflows use AI judgment with tightly scoped safe-outputs, threat detection, and a conservative close-with-reopen-invitation approach.
Changes:
- Rewrites
issue-triage.mdto addclose-issuecapability,roles: all/min-integrity: none(so the agent can see non-collaborator issues including spam), a restricted label taxonomy, and a customized prompt with Copilot-fix suitability assessment - Adds two new agentic workflows:
issue-comment-handler.md(processes comments to manage feedback loops, confirm fixes, and detect duplicates) andstale-closer.md(weekly AI-driven backlog cleanup capped at 10 closures) - Deletes the old
close-stale-issues.yml(hourlyactions/stale) and upgrades gh-aw from v0.49.5 to v0.79.8 with new compiled lock files and anactions-lock.json
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/issue-triage.md |
Rewritten triage prompt: adds close-issue, roles: all, min-integrity: none, label taxonomy restrictions, repo-specific prompt with Copilot-fix suitability assessment |
.github/workflows/issue-triage.lock.yml |
Regenerated compiled lock file for issue-triage (gh-aw v0.79.8) |
.github/workflows/issue-comment-handler.md |
New workflow: processes issue comments to handle feedback responses, fix confirmations, bumps, duplicate mentions, and auto-labeling |
.github/workflows/issue-comment-handler.lock.yml |
Compiled lock file for issue-comment-handler |
.github/workflows/stale-closer.md |
New workflow: weekly AI-driven stale issue closer (max 10/week, conservative) |
.github/workflows/stale-closer.lock.yml |
Compiled lock file for stale-closer |
.github/workflows/close-stale-issues.yml |
Deleted: old hourly actions/stale workflow replaced by stale-closer.md |
.github/aw/actions-lock.json |
New: pinned action SHAs for gh-aw v0.79.8 |
Comment on lines
+1
to
+10
| --- | ||
| description: | | ||
| Periodic intelligent stale-issue closer for PowerShell/vscode-powershell. Runs | ||
| weekly to identify open issues where we realistically cannot make progress — | ||
| long-standing reports lacking reproduction steps, duplicate requests after the | ||
| original was closed, external blockers with no path forward, or feature requests | ||
| that don't align with project direction. Uses AI judgment instead of time-based | ||
| rules. Conservative: only closes when confidence is high that keeping it open | ||
| won't help anyone. | ||
|
|
JustinGrote
approved these changes
Jun 18, 2026
JustinGrote
left a comment
Collaborator
There was a problem hiding this comment.
LGTM I guess, we will just have to see if it works because we don't exactly have a "Test" environment for this, but I don't see anything that's irreversible if it goes nuts.
Member
Author
|
@JustinGrote yeah this is highly experimental buuut if it even just catches and closes our spam issues I would be happy. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes and customizes the agentic issue triage workflow originally added in #5409 (which was broken and never customized), then expands it into three focused workflows that keep the issue backlog clean with minimal maintainer effort.
What was broken
The #5409 workflow couldn't function:
roles: [admin, maintainer, write]+min-integrity: approvedsilently dropped issues from non-collaborators (exactly where spam comes from)add-labels/add-commentwere configured, noclose-issueCOPILOT_GITHUB_TOKENsecretThe three workflows
1.
issue-triage.md— Incoming triage (on issue open/reopen)Needs: Triage+ area/platform/type labels2.
issue-comment-handler.md— Smart comment processor (onissue_comment)Needs: Author Feedback→Needs: TriageResolution-Duplicatefor verificationArea-*/OS-*/Bug-*labels3.
stale-closer.md— Intelligent backlog cleanup (weekly)actions/staletime-based rulesOther changes
close-stale-issues.yml(Issue Housekeeping) — itsactions/staleapproach didn't work well; replaced bystale-closer.mdpermissions: copilot-requests: writeon all three — no PAT, centralized org Copilot billing, tokens minted/revoked per runactions-lock.jsonand all.lock.ymlfiles)agentics-maintenance.yml— none of our workflows useexpiresThe end goal
Maintainers filter to
Needs: Triageand see only real, actionable, pre-labeled issues. Spam and duplicates are gone within minutes, verified fixes close themselves, and the stale backlog shrinks gradually under conservative AI judgment.Notes
gh aw compile: 0 errors, 0 warnings).lock.ymlfiles are committed for transparency (GitHub Actions runs them directly). Prompt-only edits to the.mdbody take effect at runtime without recompiling; only frontmatter changes requiregh aw compile.staged: trueorgh aw trialdry-run before relying on auto-close behavior in production.Drafted by Copilot (Claude Opus 4.8).