Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit a73668a

Browse filesBrowse files
[3.14] gh-118150: warn in doc about results asymmetry for difflib junk (GH-153892) (GH-154659)
(cherry picked from commit 59e67c2) Co-authored-by: Lenormand Julien <lenormand.julien0@gmail.com>
1 parent aba6bb9 commit a73668a
Copy full SHA for a73668a

2 files changed

+14-17Lines changed: 14 additions & 17 deletions

File tree

Expand file treeCollapse file tree
Open diff view settings
Filter options
Expand file treeCollapse file tree
Open diff view settings
Collapse file

‎Doc/library/difflib.rst‎

Copy file name to clipboardExpand all lines: Doc/library/difflib.rst
+12-17Lines changed: 12 additions & 17 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -44,13 +44,18 @@ diffs. For comparing directories and files, see also, the :mod:`filecmp` module.
4444
complicated way on how many elements the sequences have in common; best case
4545
time is linear.
4646

47-
**Automatic junk heuristic:** :class:`SequenceMatcher` supports a heuristic that
48-
automatically treats certain sequence items as junk. The heuristic counts how many
49-
times each individual item appears in the sequence. If an item's duplicates (after
50-
the first one) account for more than 1% of the sequence and the sequence is at least
51-
200 items long, this item is marked as "popular" and is treated as junk for
52-
the purpose of sequence matching. This heuristic can be turned off by setting
53-
the ``autojunk`` argument to ``False`` when creating the :class:`SequenceMatcher`.
47+
**Junk**: :class:`SequenceMatcher` accepts an ``isjunk`` predicate and an
48+
``autojunk`` flag. Items that are considered as junk will not be considered
49+
to find similar content blocks. This can produce better results for humans
50+
(typically breaking on whitespace) and faster (because it reduces the number
51+
of possible combinations). But it can also cause pathological cases where
52+
too many items considered junk cause an unexpectedly large (but correct)
53+
diff result.
54+
You should consider tuning them or turning them off depending on your data.
55+
Moreover, only the second sequence is inspected for junk. This causes the diff
56+
output to not be symmetrical.
57+
When ``autojunk=True``, it will consider as junk the items that account for more
58+
than 1% of the sequence, if it is at least 200 items long.
5459

5560
.. versionchanged:: 3.2
5661
Added the *autojunk* parameter.
@@ -554,16 +559,6 @@ The :class:`SequenceMatcher` class has this constructor:
554559
to try :meth:`quick_ratio` or :meth:`real_quick_ratio` first to get an
555560
upper bound.
556561

557-
.. note::
558-
559-
Caution: The result of a :meth:`ratio` call may depend on the order of
560-
the arguments. For instance::
561-
562-
>>> SequenceMatcher(None, 'tide', 'diet').ratio()
563-
0.25
564-
>>> SequenceMatcher(None, 'diet', 'tide').ratio()
565-
0.5
566-
567562

568563
.. method:: quick_ratio()
569564

Collapse file
+2Lines changed: 2 additions & 0 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Clarify in the :mod:`difflib` documentation what *junk* actually does, its
2+
drawbacks, and how to control it.

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.