perf: skip dependency parser during POS tagging#4409
Draft
KRRT7 wants to merge 6 commits into
Unstructured-IO:mainUnstructured-IO/unstructured:mainfrom
KRRT7:perf-parser-free-pos-stackedKRRT7/unstructured:perf-parser-free-pos-stackedCopy head branch name to clipboard
Draft
perf: skip dependency parser during POS tagging#4409KRRT7 wants to merge 6 commits intoUnstructured-IO:mainUnstructured-IO/unstructured:mainfrom KRRT7:perf-parser-free-pos-stackedKRRT7/unstructured:perf-parser-free-pos-stackedCopy head branch name to clipboard
KRRT7 wants to merge 6 commits into
Unstructured-IO:mainUnstructured-IO/unstructured:mainfrom
KRRT7:perf-parser-free-pos-stackedKRRT7/unstructured:perf-parser-free-pos-stackedCopy head branch name to clipboard
Conversation
KRRT7
force-pushed
the
perf-parser-free-pos-stacked
branch
from
July 17, 2026 03:39
35d48a9 to
11d518e
Compare
Contributor
|
LGTM. will wait for #4408 to land before formal approval |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack dependency
This draft is stacked on #4408 and should be reviewed and merged after that PR. Once #4408 lands, this branch will be rebased onto
mainso the final diff contains only the parser-free POS change.Summary
Skip spaCy's dependency parser in
pos_tag()while retaining thetok2vecandtaggercomponents that producetoken.tag_.The dependency parser runs after the tagger, and
pos_tag()immediately discards its output. Disabling it removes unnecessary statistical work without changing returned POS tags.Performance
Benchmarked against the tokenizer-only parent branch on an 8-vCPU AMD EPYC host with 15 GiB RAM and Python 3.12. The spaCy model was warmed, NLP result caches were cleared before every iteration, and each full-matrix result is the median of three runs.
A seven-iteration follow-up on the short workloads measured HTML 1.5% faster and PPTX 1.6% slower, confirming both are effectively neutral. The repeatable gains are in the text-heavy TXT and DOCX paths.
Compared with
upstream/main, the combined tokenizer-only and parser-free-POS stack improves the representative aggregate from 85.801s to 70.998s: approximately 17.3% faster / 1.21x.Output parity was exact for all four documents.
Testing
pos_tag()disables the parser.