fix(web-cmdlets): use response charset instead of ASCII in WebResponseObject.ToString() - #27742
#27742Open
sergioperezcheco wants to merge 1 commit into
PowerShell:masterPowerShell/PowerShell:masterfrom
sergioperezcheco:fix/webresponse-tostring-encodingsergioperezcheco/PowerShell:fix/webresponse-tostring-encodingCopy head branch name to clipboard
Open
fix(web-cmdlets): use response charset instead of ASCII in WebResponseObject.ToString()#27742sergioperezcheco wants to merge 1 commit intoPowerShell:masterPowerShell/PowerShell:masterfrom sergioperezcheco:fix/webresponse-tostring-encodingsergioperezcheco/PowerShell:fix/webresponse-tostring-encodingCopy head branch name to clipboard
sergioperezcheco wants to merge 1 commit into
PowerShell:masterPowerShell/PowerShell:masterfrom
sergioperezcheco:fix/webresponse-tostring-encodingsergioperezcheco/PowerShell:fix/webresponse-tostring-encodingCopy head branch name to clipboard
Conversation
…eObject.ToString() ToString() hardcoded Encoding.ASCII to decode the response body bytes, which silently replaced any non-ASCII character with '?' (or '.'). This affected both the user-facing .ToString() call and the RawContent property that feeds into it. Use the character set declared in the response Content-Type header, falling back to UTF-8 (the default used by the rest of the web cmdlets) when no charset is present. UTF-8 is a strict superset of ASCII, so pure-ASCII responses are unaffected. Resolves PowerShell#27154. Signed-off-by: sergioperezcheco <checo520@outlook.com>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes WebResponseObject.ToString() decoding so response bodies with non-ASCII characters are preserved by using the response Content-Type charset (falling back to UTF-8), addressing issue #27154 and improving RawContent fidelity.
Changes:
- Update
WebResponseObject.ToString()to resolve the encoding from the response charset viaWebResponseHelper.GetCharacterSet+StreamHelper.TryGetEncoding(defaulting to UTF-8). - Update the
InitializeRawContentcomment to reflect the new encoding behavior. - Add a Pester test validating non-ASCII preservation for a UTF-8 response body.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/Microsoft.PowerShell.Commands.Utility/commands/utility/WebCmdlet/Common/WebResponseObject.Common.cs |
Switches ToString() decoding from hardcoded ASCII to response charset/default UTF-8 and updates a related comment. |
test/powershell/Modules/Microsoft.PowerShell.Utility/WebCmdlets.Tests.ps1 |
Adds a regression test ensuring Invoke-WebRequest preserves non-ASCII characters through .ToString(). |
Comment on lines
+187
to
+193
| // Decode using the response's character set when available so non-ASCII | ||
| // content is preserved. Fall back to UTF-8, which is a safe superset of | ||
| // ASCII and the default encoding used elsewhere in the web cmdlets. | ||
| string? characterSet = WebResponseHelper.GetCharacterSet(BaseResponse); | ||
| StreamHelper.TryGetEncoding(characterSet, out Encoding encoding); | ||
|
|
||
| char[] stringContent = encoding.GetChars(Content); |
Comment on lines
+1537
to
+1549
| It "Verifies Invoke-WebRequest ToString preserves non-ASCII characters when charset is declared" { | ||
| $query = @{ | ||
| contenttype = 'text/plain; charset=utf-8' | ||
| body = 'café' | ||
| } | ||
| $uri = Get-WebListenerUrl -Test 'Response' -Query $query | ||
| $response = ExecuteWebRequest -Uri $uri -UseBasicParsing | ||
|
|
||
| $response.Error | Should -BeNullOrEmpty | ||
| # ToString() should preserve the non-ASCII 'é' instead of replacing it with '?'. | ||
| $response.Output.ToString() | Should -Match 'café' | ||
| } | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WebResponseObject.ToString() hardcoded Encoding.ASCII to decode the response body bytes, so any non-ASCII character came out as '?' (or '.' after the printable-char filter). This affects both the direct .ToString() call and the RawContent property that is built from it.
This change resolves the encoding from the response Content-Type charset via WebResponseHelper.GetCharacterSet / StreamHelper.TryGetEncoding, falling back to UTF-8 (ContentHelper.GetDefaultEncoding) when no charset is declared. UTF-8 is a strict superset of ASCII, so existing pure-ASCII responses are unchanged. I also updated the now-stale 'Use ASCII encoding' comment in InitializeRawContent.
Added a Pester test that serves a UTF-8 body containing 'café' and asserts the non-ASCII character survives ToString().
Resolves #27154.