[release/v7.5] Mark flaky Update-Help web tests as pending to unblock CI#26837
[release/v7.5] Mark flaky Update-Help web tests as pending to unblock CI#26837daxian-dbw merged 2 commits intoPowerShell:release/v7.5PowerShell/PowerShell:release/v7.5from daxian-dbw:backport/release/v7.5/26796-6b8d724b0daxian-dbw/PowerShell:backport/release/v7.5/26796-6b8d724b0Copy head branch name to clipboard
Conversation
There was a problem hiding this comment.
Pull request overview
Backport to release/v7.5 to reduce CI flakiness by marking web-dependent Update-Help tests as Pending while keeping them visible for future re-enablement.
Changes:
- Add
-MarkAsPendingswitch toRunUpdateHelpTestsand useSet-ItResult -Pendingfor CI-tagged web Update-Help tests. - Add defensive null-guards around cleanup paths (help folder removal; Save-Help temp folder removal).
- Add a repository instruction doc describing the required
Set-ItResult+returnpattern in Pester tests.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| test/powershell/engine/Help/UpdatableHelpSystem.Tests.ps1 | Marks CI web Update-Help tests pending; adds null-guards around cleanup/temp paths. |
| .github/instructions/pester-set-itresult-pattern.instructions.md | Documents the Set-ItResult (Pending/Skipped) pattern and the required return. |
| ```powershell | ||
| It "Clear-Host works correctly" { | ||
| if ($IsARM64) { | ||
| Set-ItResult -Pending -Because "ARM64 runs in non-interactively mode and Clear-Host does not work." |
There was a problem hiding this comment.
Spelling/grammar: "non-interactively mode" should be "non-interactive mode".
| Set-ItResult -Pending -Because "ARM64 runs in non-interactively mode and Clear-Host does not work." | |
| Set-ItResult -Pending -Because "ARM64 runs in non-interactive mode and Clear-Host does not work." |
| $saveHelpFolder = if ($TestDrive) { | ||
| Join-Path $TestDrive (Get-Random).ToString() | ||
| } else { | ||
| $null | ||
| } | ||
|
|
||
| if ($saveHelpFolder) { | ||
| New-Item $saveHelpFolder -Force -ItemType Directory > $null | ||
| } |
There was a problem hiding this comment.
$saveHelpFolder is computed in the RunSaveHelpTests discovery path (outside the It block) using $TestDrive. If $TestDrive isn't populated during discovery (common in Pester), this ends up capturing $null into the It block and Save-Help -DestinationPath $saveHelpFolder will fail when the test actually runs. Consider moving the folder creation into the It block (or a BeforeAll/BeforeEach inside the Describe/Context) and cleaning it up in AfterEach/AfterAll so it always uses the runtime $TestDrive.
…#26796) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: TravisEz13 <10873629+TravisEz13@users.noreply.github.com>
edaf509 to
a8dbf0b
Compare
Backport of #26796 to release/v7.5
Triggered by @daxian-dbw on behalf of @app/copilot-swe-agent
Original CL Label: CL-Test
/cc @PowerShell/powershell-maintainers
Impact
REQUIRED: Choose either Tooling Impact or Customer Impact (or both). At least one checkbox must be selected.
Tooling Impact
Customer Impact
Update-Help tests downloading from the web were causing intermittent CI failures due to network issues (timeouts, task cancellations). Tests are now marked as pending to unblock CI while keeping them visible for future resolution. Issues #2807 and #6541 document the pattern.
Regression
REQUIRED: Check exactly one box.
This is not a regression.
Testing
33 of 38 tests now marked as NotRun (pending), unblocking CI while keeping tests visible for future resolution. Tests using local assets (-SourcePath variants) remain active. Null checks prevent issues during Pester discovery phase.
Risk
REQUIRED: Check exactly one box.
Changes only affect test execution flow by marking flaky web-dependent tests as pending. Does not modify product code. Adds defensive null checks to handle Pester discovery phase gracefully. Only affects CI-tagged tests; Feature-tagged tests remain active.