Closed
Description
From reading the invocation log, there seems to be many datasets. My suspicion is that now the project has many "datasets" which makes list_datasets to be more flaky. I think the function is fetching pages and pages under the cover.
I think manually deleting those datasets will greatly mitigate
Log:
Traceback (most recent call last):
File "/tmpfs/src/github/python-docs-samples/datalabeling/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
return callable_(*args, **kwargs)
File "/tmpfs/src/github/python-docs-samples/datalabeling/.nox/py-3-7/lib/python3.7/site-packages/grpc/_channel.py", line 826, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/tmpfs/src/github/python-docs-samples/datalabeling/.nox/py-3-7/lib/python3.7/site-packages/grpc/_channel.py", line 729, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.DEADLINE_EXCEEDED
details = "Deadline Exceeded"
debug_error_string = "{"created":"@1588849942.771695832","description":"Error received from peer ipv4:108.177.98.81:443","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Deadline Exceeded","grpc_status":4}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/tmpfs/src/github/python-docs-samples/datalabeling/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/retry.py", line 184, in retry_target
return target()
File "/tmpfs/src/github/python-docs-samples/datalabeling/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/timeout.py", line 214, in func_with_timeout
return func(*args, **kwargs)
File "/tmpfs/src/github/python-docs-samples/datalabeling/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "<string>", line 3, in raise_from
google.api_core.exceptions.DeadlineExceeded: 504 Deadline Exceeded
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/tmpfs/src/github/python-docs-samples/datalabeling/manage_dataset_test.py", line 72, in test_list_dataset
run_sample()
File "/tmpfs/src/github/python-docs-samples/datalabeling/.nox/py-3-7/lib/python3.7/site-packages/backoff/_sync.py", line 94, in retry
ret = target(*args, **kwargs)
File "/tmpfs/src/github/python-docs-samples/datalabeling/manage_dataset_test.py", line 70, in run_sample
manage_dataset.list_datasets(PROJECT_ID)
File "/tmpfs/src/github/python-docs-samples/datalabeling/manage_dataset.py", line 73, in list_datasets
for element in response:
File "/tmpfs/src/github/python-docs-samples/datalabeling/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/page_iterator.py", line 212, in _items_iter
for page in self._page_iter(increment=False):
File "/tmpfs/src/github/python-docs-samples/datalabeling/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/page_iterator.py", line 249, in _page_iter
page = self._next_page()
File "/tmpfs/src/github/python-docs-samples/datalabeling/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/page_iterator.py", line 534, in _next_page
response = self._method(self._request)
File "/tmpfs/src/github/python-docs-samples/datalabeling/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py", line 143, in __call__
return wrapped_func(*args, **kwargs)
File "/tmpfs/src/github/python-docs-samples/datalabeling/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/retry.py", line 286, in retry_wrapped_func
on_error=on_error,
File "/tmpfs/src/github/python-docs-samples/datalabeling/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/retry.py", line 206, in retry_target
last_exc,
File "<string>", line 3, in raise_from
google.api_core.exceptions.RetryError: Deadline of 600.0s exceeded while calling functools.partial(<function _wrap_unary_errors.<locals>.error_remapped_callable at 0x7f44b77f1320>, parent: "projects/python-docs-samples-tests"
page_token: "2020-02-26T00:23:46.026853412-08:00"
, metadata=[('x-goog-request-params', 'parent=projects/python-docs-samples-tests'), ('x-goog-api-client', 'gl-python/3.7.5 grpc/1.28.1 gax/1.17.0 gapic/0.4.0')]), last exception: 504 Deadline Exceeded
Metadata
Metadata
Assignees
Labels
Issues related to the AI Platform Data Labeling Service API.Issues related to the AI Platform Data Labeling Service API.Important issue which blocks shipping the next release. Will be fixed prior to next release.Important issue which blocks shipping the next release. Will be fixed prior to next release.Error or flaw in code with unintended results or allowing sub-optimal usage patterns.Error or flaw in code with unintended results or allowing sub-optimal usage patterns.