Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit 89679ce

Browse filesBrowse files
authored
Merge pull request python-gitlab#678 from appian/backoff-requests
Fix missing "Retry-After" header and fix integration tests
2 parents ce2c835 + 16bda20 commit 89679ce
Copy full SHA for 89679ce

File tree

Expand file treeCollapse file tree

3 files changed

+27
-5
lines changed
Open diff view settings
Filter options
Expand file treeCollapse file tree

3 files changed

+27
-5
lines changed
Open diff view settings
Collapse file

‎docs/api-usage.rst‎

Copy file name to clipboardExpand all lines: docs/api-usage.rst
+15-1Lines changed: 15 additions & 1 deletion
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -299,7 +299,9 @@ Rate limits
299299

300300
python-gitlab obeys the rate limit of the GitLab server by default. On
301301
receiving a 429 response (Too Many Requests), python-gitlab sleeps for the
302-
amount of time in the Retry-After header that GitLab sends back.
302+
amount of time in the Retry-After header that GitLab sends back. If GitLab
303+
does not return a response with the Retry-After header, python-gitlab will
304+
perform an exponential backoff.
303305

304306
If you don't want to wait, you can disable the rate-limiting feature, by
305307
supplying the ``obey_rate_limit`` argument.
@@ -312,6 +314,18 @@ supplying the ``obey_rate_limit`` argument.
312314
gl = gitlab.gitlab(url, token, api_version=4)
313315
gl.projects.list(all=True, obey_rate_limit=False)
314316
317+
If you do not disable the rate-limiting feature, you can supply a custom value
318+
for ``max_retries``; by default, this is set to 10. To retry without bound when
319+
throttled, you can set this parameter to -1. This parameter is ignored if
320+
``obey_rate_limit`` is set to ``False``.
321+
322+
.. code-block:: python
323+
324+
import gitlab
325+
import requests
326+
327+
gl = gitlab.gitlab(url, token, api_version=4)
328+
gl.projects.list(all=True, max_retries=12)
315329
316330
.. warning::
317331

Collapse file

‎gitlab/__init__.py‎

Copy file name to clipboardExpand all lines: gitlab/__init__.py
+11-3Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -477,6 +477,10 @@ def http_request(self, verb, path, query_data={}, post_data=None,
477477
# obey the rate limit by default
478478
obey_rate_limit = kwargs.get("obey_rate_limit", True)
479479

480+
# set max_retries to 10 by default, disable by setting it to -1
481+
max_retries = kwargs.get("max_retries", 10)
482+
cur_retries = 0
483+
480484
while True:
481485
result = self.session.send(prepped, timeout=timeout, **settings)
482486

@@ -486,9 +490,13 @@ def http_request(self, verb, path, query_data={}, post_data=None,
486490
return result
487491

488492
if 429 == result.status_code and obey_rate_limit:
489-
wait_time = int(result.headers["Retry-After"])
490-
time.sleep(wait_time)
491-
continue
493+
if max_retries == -1 or cur_retries < max_retries:
494+
wait_time = 2 ** cur_retries * 0.1
495+
if "Retry-After" in result.headers:
496+
wait_time = int(result.headers["Retry-After"])
497+
cur_retries += 1
498+
time.sleep(wait_time)
499+
continue
492500

493501
error_message = result.content
494502
try:
Collapse file

‎tools/python_test_v4.py‎

Copy file name to clipboardExpand all lines: tools/python_test_v4.py
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -798,7 +798,7 @@
798798
except gitlab.GitlabCreateError as e:
799799
error_message = e.error_message
800800
break
801-
assert 'Retry later' in error_message.decode()
801+
assert 'Retry later' in error_message
802802
[current_project.delete() for current_project in projects]
803803
settings.throttle_authenticated_api_enabled = False
804804
settings.save()

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.