Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit 59e67c2

Browse filesBrowse files
authored
gh-118150: warn in doc about results asymmetry for difflib junk (GH-153892)
1 parent 9560bd8 commit 59e67c2
Copy full SHA for 59e67c2

2 files changed

+14-17Lines changed: 14 additions & 17 deletions

File tree

Expand file treeCollapse file tree
Open diff view settings
Filter options
Expand file treeCollapse file tree
Open diff view settings
Collapse file

‎Doc/library/difflib.rst‎

Copy file name to clipboardExpand all lines: Doc/library/difflib.rst
+12-17Lines changed: 12 additions & 17 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -40,13 +40,18 @@ diffs. For comparing directories and files, see also, the :mod:`filecmp` module.
4040
complicated way on how many elements the sequences have in common; best case
4141
time is linear.
4242

43-
**Automatic junk heuristic:** :class:`SequenceMatcher` supports a heuristic that
44-
automatically treats certain sequence items as junk. The heuristic counts how many
45-
times each individual item appears in the sequence. If an item's duplicates (after
46-
the first one) account for more than 1% of the sequence and the sequence is at least
47-
200 items long, this item is marked as "popular" and is treated as junk for
48-
the purpose of sequence matching. This heuristic can be turned off by setting
49-
the ``autojunk`` argument to ``False`` when creating the :class:`SequenceMatcher`.
43+
**Junk**: :class:`SequenceMatcher` accepts an ``isjunk`` predicate and an
44+
``autojunk`` flag. Items that are considered as junk will not be considered
45+
to find similar content blocks. This can produce better results for humans
46+
(typically breaking on whitespace) and faster (because it reduces the number
47+
of possible combinations). But it can also cause pathological cases where
48+
too many items considered junk cause an unexpectedly large (but correct)
49+
diff result.
50+
You should consider tuning them or turning them off depending on your data.
51+
Moreover, only the second sequence is inspected for junk. This causes the diff
52+
output to not be symmetrical.
53+
When ``autojunk=True``, it will consider as junk the items that account for more
54+
than 1% of the sequence, if it is at least 200 items long.
5055

5156
.. versionchanged:: 3.2
5257
Added the *autojunk* parameter.
@@ -558,16 +563,6 @@ The :class:`SequenceMatcher` class has this constructor:
558563
to try :meth:`quick_ratio` or :meth:`real_quick_ratio` first to get an
559564
upper bound.
560565

561-
.. note::
562-
563-
Caution: The result of a :meth:`ratio` call may depend on the order of
564-
the arguments. For instance::
565-
566-
>>> SequenceMatcher(None, 'tide', 'diet').ratio()
567-
0.25
568-
>>> SequenceMatcher(None, 'diet', 'tide').ratio()
569-
0.5
570-
571566

572567
.. method:: quick_ratio()
573568

Collapse file
+2Lines changed: 2 additions & 0 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Clarify in the :mod:`difflib` documentation what *junk* actually does, its
2+
drawbacks, and how to control it.

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.