Bugfix for #15 for compressing large files#16
Merged
allanlei merged 1 commit intoallanlei:masterallanlei/python-zipstream:masterfrom May 6, 2016
Merged
Bugfix for #15 for compressing large files#16allanlei merged 1 commit intoallanlei:masterallanlei/python-zipstream:masterfrom
allanlei merged 1 commit intoallanlei:masterallanlei/python-zipstream:masterfrom
Conversation
This is a potential quick fix for Issue allanlei#15 which occurs when attempting to compress files larger than ZIP64_LIMIT. `zinfo.file_size` is never initialized to the correct file size and thus the determination of whether zip64 is required is made based on the file size of 0. This later results in an exception being raised as though the file size increased during compression, since the file size is actually counted during compression and later saved over `zinfo.file_size`. It is important to note that this fix may not be cross platform. Different versions of Python do different things on Windows with the `st_size` parameter in the `os.stat` call. So, that may be worth investigating further. However, in the short term, this will fix the problem on Linux, Mac, and some Windows platforms without making it worse where it still doesn't work. I would leave it to the maintainers to make a broader decision on whether this fix is appropriate or if a better solution would be desired. I'm happy to help.
Author
|
Alternatively, replacing |
|
This fix works if st[6] is replaced with os.path.getsize(filename), as shown above, and allowZip64=True is passed as an argument. I was able to download and decompress a 2.5GB file that was previously failing to unzip. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a potential quick fix for Issue #15 which occurs when attempting to compress files larger than ZIP64_LIMIT.
zinfo.file_sizeis never initialized to the correct file size and thus the determination of whether zip64 is required is made based on the file size of 0. This later results in an exception being raised as though the file size increased during compression, since the file size is actually counted during compression and later saved overzinfo.file_size.It is important to note that this fix may not be cross platform. Different versions of Python do different things on Windows with the
st_sizeparameter in theos.statcall. So, that may be worth investigating further. However, in the short term, this will fix the problem on Linux, Mac, and some Windows platforms without making it worse where it still doesn't work.I would leave it to the maintainers to make a broader decision on whether this fix is appropriate or if a better solution would be desired. I'm happy to help.