Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

base64.b85encode uses significant amount of RAM #101178

Copy link
Copy link
@AbdealiLoKo

Description

@AbdealiLoKo
Issue body actions

Bug report

On the same string:

  • b64encode: RAM: 604MB, CPU: 79%, Time taken: 1.18sec and gave a result
  • b85encode: RAM: 6.9GB, CPU: 64%, Time taken: 45sec and crashed due to insufficient RAM (most likely)

b85encode takes up my entire RAM and crashes

On IPython:

In [1]: import base64

In [2]: contents = b'a' * 250 * 1024 * 1024

In [3]: %time len(base64.b64encode(contents).decode("ascii")) / 1024 / 1024
CPU times: user 489 ms, sys: 414 ms, total: 904 ms
Wall time: 974 ms
Out[3]: 333.33333587646484

In [4]: %time len(base64.b85encode(contents).decode("ascii")) / 1024 / 1024
Killed

Here is the GNU time stats:

$ /usr/bin/time /opt/python3.9/bin/python -c "import base64; print(len(base64.b64encode(b'a' * 250 * 1024 * 1024)) / 1024 / 1024)"
333.33333587646484
0.55user 0.39system 0:01.18elapsed 79%CPU (0avgtext+0avgdata 604008maxresident)k
0inputs+0outputs (0major+151155minor)pagefaults 0swaps

$ /usr/bin/time /opt/python3.9/bin/python -c "import base64; print(len(base64.b85encode(b'a' * 250 * 1024 * 1024)) / 1024 / 1024)"
26.08user 3.39system 0:45.86elapsed 64%CPU (0avgtext+0avgdata 6966080maxresident)k
512inputs+0outputs (2major+1756895minor)pagefaults 0swaps

I have gotten same results in Python 3.6, 3.7, 3.8, 3.9

Your environment

  • CPython versions tested on: Python 3.9 installed with miniconda
  • Operating system and architecture: CentOS 7, AWS t3.large machine

Linked PRs

Reactions are currently unavailable

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance or resource usagePerformance or resource usagestdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Morty Proxy This is a proxified and sanitized view of the page, visit original site.