Keep track of offsets within substreams where possible #1015
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes construct#1014 This patch wraps the BytesIO substreams made by the following subconstructs during parsing: * OffsettedEnd * Prefixed * FixedSize * NullTerminated * NullStripped * ProcessXor This allows the Tell construct to return offsets that are relative to the beginning of the stream, rather than relative to the last occurence of these subconstructs. This has not been implemented for the following subconstructs: * Transformed: there is no guarantee that there won't be any byte swapping in the transformation, therefore we can't map offsets without knowledge of the specific transformation. * ProcessRotateLeft: tell() offsets would be decreasing, which could cause all sorts of problems. In both those cases, it makes more sense for the user to use a Tell immediately before the Transformed or ProcessRotateLeft and do their own mapping.
7786978 to
5ef489b
Compare
Member
|
Merging it now. I think we could even go one step further and implement a SubBytesIO that reimplements reads instead of rereading however much data into memory. Anyway, big thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #1014
This patch wraps the BytesIO substreams made by the following subconstructs during parsing:
This allows the Tell construct to return offsets that are relative to the beginning of the stream, rather than relative to the last occurence of these subconstructs.
This has not been implemented for the following subconstructs:
Transformed: there is no guarantee that there won't be any byte swapping in the transformation, therefore we can't map offsets without knowledge of the specific transformation.
ProcessRotateLeft: tell() offsets would be decreasing, which could cause all sorts of problems.
In both those cases, it makes more sense for the user to use a Tell immediately before the Transformed or ProcessRotateLeft and do their own mapping.