Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Conversation

@aag
Copy link
Contributor

@aag aag commented Jan 9, 2013

I want to download some large files (1+ GB) from a Shotgun server. With the download_attachment() function, the whole file must be stored in memory before it can be saved to the disk. This is a problem because there might not be that much RAM available on the system I'm running the script on.

This commit adds a new function, download_attachment_to_file(), which streams the attachment directly to a file. It uses very little memory, no matter how large the file is.

I have tested the changes in Python 2.4.6 and 2.7.3, in Linux and Windows.

I'm not sure what the best way is to submit documentation changes, so here's a gist with Markdown documentation for the Methods Reference page on the wiki:
https://gist.github.com/4490286

This new function downloads an attachment directly to disk instead of to
memory.  It is useful for two reasons:

* If the user wants to save the attachment to a file anyway, it turns
  several lines of user code into one line.

* If the file is very large, storing the attachment in memory before
  writing it to a file can fill all the available memory in the system.
@kporangehat
Copy link

Thanks Adam. That's a good idea. We'll take a look shortly and see how it shakes out. Thanks too for the doc contribution (always nice to save some time!).

cheers,
kp

@rocktavious
Copy link

Whats the status of this? Seems like its been a while since this has been looked at and i can not find anything in the API that would have replaced this. Can we get this merged into the main codebase?

@mattdaw
Copy link
Contributor

mattdaw commented Jun 12, 2013

Hi, sorry for the delay, we lost track of this one. We'll discuss the approach in our pipeline meeting tomorrow and report back.

Matt

@kporangehat
Copy link

We talked through this today and have scheduled it for our next sprint starting next week. We want to make a couple of minor adjustments, but overall it looks good. Thanks for the pull request. We'll update you next week.

cheers,
kp

@mattdaw
Copy link
Contributor

mattdaw commented Jun 22, 2013

Hey Brandon, could you please code review and merge? Thanks!

brandonvfx added a commit that referenced this pull request Jun 25, 2013
…wnload_attachment

22152 support hash for download attachment (and Pull #32 write directly to disk)
@kporangehat
Copy link

We've tagged and released v3.0.14 which includes a modified version of this pull request. We opted to have it as an optional parameter on the existing download_attachment() in order to keep things concise. The docs have been updated.

Thanks a lot for submitting this! (we gave you credit in the Changelog).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.