Zero-width characters cause unbounded memory use in PyREPL

Bug report

Bug description:

Trying to paste zero-width characters (U+200B, U+200C, U+200D, U+FEFF) into the new REPL causes it to get stuck and keep allocating memory. Python 3.12 handles them fine.

This seems to happen during screen calculation, so the amount of characters needed to reproduce will depend on the number of cells in a terminal emulator window (I got it working with as few as 3 with a big enough zoom, but 100 should be enough).

A table with all of them x100 to copy for convenience:

ZWSP	ZWNJ	ZWJ	ZWNBSP
	`‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌`	`‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍`

To reproduce

Open the new REPL
Paste in some zero-width characters
The REPL should get stuck and keep allocating memory
Can be stopped with double ^C or a single ^\

The culprit seems to be this loop in Lib/_pyrepl/reader.py:Reader.calc_screen and its screen/screen_info appends:

cpython/Lib/_pyrepl/reader.py

Lines 388 to 411 in f4e5643

    
           while l: 
        
               prelen = lp if i == 0 else 0 
        
               index_to_wrap_before = 0 
        
               column = 0 
        
               for character_width in l2: 
        
                   if column + character_width >= self.console.width - prelen: 
        
                       break 
        
                   index_to_wrap_before += 1 
        
                   column += character_width 
        
               pre = prompt if i == 0 else "" 
        
               if len(l) > index_to_wrap_before: 
        
                   offset += index_to_wrap_before 
        
                   post = "\\" 
        
                   after = [1] 
        
               else: 
        
                   offset += index_to_wrap_before + 1  # Takes the newline 
        
                   post = "" 
        
                   after = [] 
        
               last_refresh_line_end_offsets.append(offset) 
        
               screen.append(pre + l[:index_to_wrap_before] + post) 
        
               screeninfo.append((prelen, l2[:index_to_wrap_before] + after)) 
        
               l = l[index_to_wrap_before:] 
        
               l2 = l2[index_to_wrap_before:] 
        
               i += 1

An additional demo

Screen.Recording.2024-11-11.at.13.52.32.mov

CPython versions tested on:

3.12, 3.13, CPython main branch

Operating systems tested on:

Linux, macOS

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Zero-width characters cause unbounded memory use in PyREPL #126685

Bug report

Bug description:

To reproduce

An additional demo

CPython versions tested on:

Operating systems tested on:

Metadata

Assignees

Labels

Fields

Projects

Milestone

Relationships

Development

	while l:
	prelen = lp if i == 0 else 0
	index_to_wrap_before = 0
	column = 0
	for character_width in l2:
	if column + character_width >= self.console.width - prelen:
	break
	index_to_wrap_before += 1
	column += character_width
	pre = prompt if i == 0 else ""
	if len(l) > index_to_wrap_before:
	offset += index_to_wrap_before
	post = "\\"
	after = [1]
	else:
	offset += index_to_wrap_before + 1 # Takes the newline
	post = ""
	after = []
	last_refresh_line_end_offsets.append(offset)
	screen.append(pre + l[:index_to_wrap_before] + post)
	screeninfo.append((prelen, l2[:index_to_wrap_before] + after))
	l = l[index_to_wrap_before:]
	l2 = l2[index_to_wrap_before:]
	i += 1

Search code, repositories, users, issues, pull requests...

Uh oh!

Zero-width characters cause unbounded memory use in PyREPL #126685

Description

Bug report

Bug description:

To reproduce

An additional demo

CPython versions tested on:

Operating systems tested on:

Metadata

Metadata

Assignees

Labels

Fields

Projects

Milestone

Relationships

Development

Issue actions