Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit 429754f

Browse filesBrowse files
authored
Fix unicode decode bug on surrogate error mode (#5546)
* subtract with overflow to check for whether to use surrogate * enable test_argparse for windows on ci ------ Signed-off-by: Ashwin Naren <arihant2math@gmail.com>
1 parent b4f0a58 commit 429754f
Copy full SHA for 429754f

File tree

2 files changed

+6
-3
lines changed
Filter options

2 files changed

+6
-3
lines changed

‎.github/workflows/ci.yaml

Copy file name to clipboardExpand all lines: .github/workflows/ci.yaml
-2
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@ concurrency:
1717
env:
1818
CARGO_ARGS: --no-default-features --features stdlib,zlib,importlib,encodings,sqlite,ssl
1919
# Skip additional tests on Windows. They are checked on Linux and MacOS.
20-
# test_argparse: UnicodeDecodeError
2120
# test_glob: many failing tests
2221
# test_io: many failing tests
2322
# test_os: many failing tests
@@ -26,7 +25,6 @@ env:
2625
# test_unicode: AttributeError: module '_winapi' has no attribute 'GetACP'
2726
# test_venv: couple of failing tests
2827
WINDOWS_SKIPS: >-
29-
test_argparse
3028
test_glob
3129
test_io
3230
test_os

‎vm/src/codecs.rs

Copy file name to clipboardExpand all lines: vm/src/codecs.rs
+6-1
Original file line numberDiff line numberDiff line change
@@ -619,11 +619,16 @@ fn surrogatepass_errors(err: PyObjectRef, vm: &VirtualMachine) -> PyResult<(PyOb
619619
// Not supported, fail with original exception
620620
return Err(err.downcast().unwrap());
621621
}
622+
623+
debug_assert!(range.start <= 0.max(s.len() - 1));
624+
debug_assert!(range.end >= 1.min(s.len()));
625+
debug_assert!(range.end <= s.len());
626+
622627
let mut c: u32 = 0;
623628
// Try decoding a single surrogate character. If there are more,
624629
// let the codec call us again.
625630
let p = &s.as_bytes()[range.start..];
626-
if p.len().saturating_sub(range.start) >= byte_length {
631+
if p.len().overflowing_sub(range.start).0 >= byte_length {
627632
match standard_encoding {
628633
StandardEncoding::Utf8 => {
629634
if (p[0] as u32 & 0xf0) == 0xe0

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.