Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 1f4ebb1

Browse filesBrowse files
kiraksipartheatswastchalmerlowegcf-owl-bot[bot]
authored
feat: removed pkg_resources from all test files and moved importlib into pandas extra (#1726)
* feat: Introduce compatibility with native namespace packages * Update copyright year * removed pkg_resources from all test files and moved importlib into pandas extra * feat: removed pkg_resources from all test files and moved importlib into pandas extra * Adding no cover tag to test code * reformatted with black * undo revert * perf: use the first page a results when `query(api_method="QUERY")` (#1723) * perf: use the first page a results when `query(api_method="QUERY")` * add tests * respect max_results with cached page * respect page_size, also avoid bqstorage if almost fully downloaded * skip true test if bqstorage not installed * coverage * fix: ensure query job retry has longer deadline than API request deadline (#1734) In cases where we can't disambiguate API failure from job failure, this ensures we can still retry the job at least once. * fix: `load_table_from_dataframe` now assumes there may be local null values (#1735) Even if the remote schema is REQUIRED Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [ ] Make sure to open an issue as a [bug/issue](https://togithub.com/googleapis/python-bigquery/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [ ] Ensure the tests and linter pass - [ ] Code coverage does not decrease (if any source code was changed) - [ ] Appropriate docs were updated (if necessary) Fixes #1692 🦕 * chore: standardize samples directory - delete unneeded dependencies (#1732) * chore: standardize samples directory = delete unneeded dependencies * Removed unused import for linter * fix: move grpc, proto-plus and protobuf packages to extras (#1721) * chore: move grpc, proto-plus and protobuff packages to extras * formatted with black * feat: add `job_timeout_ms` to job configuration classes (#1675) * fix: adds new property and tests * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * updates docs to correct a sphinx failure * Updates formatting * Update tests/system/test_query.py * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Update google/cloud/bigquery/job/base.py * updates one test and uses int_or_none * Update tests/system/test_query.py testing something. * Update tests/system/test_query.py * testing coverage feature * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * minor edits * tweaks to noxfile for testing purposes * add new test to base as experiment * adds a test, updates import statements * add another test * edit to tests * formatting fixes * update noxfile to correct debug code * removes unneeded comments. --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> --------- Co-authored-by: Chalmer Lowe <chalmerlowe@google.com> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Tim Swast <swast@google.com> * remove unnecessary version checks * undo bad commit, remove unneeded version checks * Revert "undo bad commit, remove unneeded version checks" This reverts commit 5c82dcf. * Revert "remove unnecessary version checks" This reverts commit 9331a7e. * revert bad changes, remove pkg_resources from file * after clarification, reimplement changes and ignore 3.12 tests * reformatted with black * removed minimum check * updated pandas installed version check --------- Co-authored-by: Anthonios Partheniou <partheniou@google.com> Co-authored-by: Tim Swast <swast@google.com> Co-authored-by: Chalmer Lowe <chalmerlowe@google.com> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
1 parent 5ce4d13 commit 1f4ebb1
Copy full SHA for 1f4ebb1

File tree

Expand file treeCollapse file tree

12 files changed

+96
-133
lines changed
Open diff view settings
Filter options
Expand file treeCollapse file tree

12 files changed

+96
-133
lines changed
Open diff view settings
Collapse file

‎.coveragerc‎

Copy file name to clipboardExpand all lines: .coveragerc
-5Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,3 @@ exclude_lines =
1212
pragma: NO COVER
1313
# Ignore debug-only repr
1414
def __repr__
15-
# Ignore pkg_resources exceptions.
16-
# This is added at the module level as a safeguard for if someone
17-
# generates the code and tries to run it without pip installing. This
18-
# makes it virtually impossible to test properly.
19-
except pkg_resources.DistributionNotFound
Collapse file

‎google/__init__.py‎

Copy file name to clipboardExpand all lines: google/__init__.py
-24Lines changed: 0 additions & 24 deletions
This file was deleted.
Collapse file

‎google/cloud/__init__.py‎

Copy file name to clipboardExpand all lines: google/cloud/__init__.py
-24Lines changed: 0 additions & 24 deletions
This file was deleted.
Collapse file

‎noxfile.py‎

Copy file name to clipboardExpand all lines: noxfile.py
+3-2Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -137,7 +137,7 @@ def mypy(session):
137137
"types-requests",
138138
"types-setuptools",
139139
)
140-
session.run("mypy", "google/cloud", "--show-traceback")
140+
session.run("mypy", "-p", "google", "--show-traceback")
141141

142142

143143
@nox.session(python=DEFAULT_PYTHON_VERSION)
@@ -149,7 +149,8 @@ def pytype(session):
149149
session.install("attrs==20.3.0")
150150
session.install("-e", ".[all]")
151151
session.install(PYTYPE_VERSION)
152-
session.run("pytype")
152+
# See https://github.com/google/pytype/issues/464
153+
session.run("pytype", "-P", ".", "google/cloud/bigquery")
153154

154155

155156
@nox.session(python=SYSTEM_TEST_PYTHON_VERSIONS)
Collapse file

‎setup.py‎

Copy file name to clipboardExpand all lines: setup.py
+2-8Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@
6262
"pandas>=1.1.0",
6363
pyarrow_dependency,
6464
"db-dtypes>=0.3.0,<2.0.0dev",
65+
"importlib_metadata>=1.0.0; python_version<'3.8'",
6566
],
6667
"ipywidgets": [
6768
"ipywidgets>=7.7.0",
@@ -108,16 +109,10 @@
108109
# benchmarks, etc.
109110
packages = [
110111
package
111-
for package in setuptools.PEP420PackageFinder.find()
112+
for package in setuptools.find_namespace_packages()
112113
if package.startswith("google")
113114
]
114115

115-
# Determine which namespaces are needed.
116-
namespaces = ["google"]
117-
if "google.cloud" in packages:
118-
namespaces.append("google.cloud")
119-
120-
121116
setuptools.setup(
122117
name=name,
123118
version=version,
@@ -143,7 +138,6 @@
143138
],
144139
platforms="Posix; MacOS X; Windows",
145140
packages=packages,
146-
namespace_packages=namespaces,
147141
install_requires=dependencies,
148142
extras_require=extras,
149143
python_requires=">=3.7",
Collapse file

‎tests/system/test_pandas.py‎

Copy file name to clipboardExpand all lines: tests/system/test_pandas.py
+10-13Lines changed: 10 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -23,9 +23,13 @@
2323
import warnings
2424

2525
import google.api_core.retry
26-
import pkg_resources
2726
import pytest
2827

28+
try:
29+
import importlib.metadata as metadata
30+
except ImportError:
31+
import importlib_metadata as metadata
32+
2933
from google.cloud import bigquery
3034

3135
from google.cloud.bigquery import enums
@@ -42,11 +46,9 @@
4246
)
4347

4448
if pandas is not None:
45-
PANDAS_INSTALLED_VERSION = pkg_resources.get_distribution("pandas").parsed_version
49+
PANDAS_INSTALLED_VERSION = metadata.version("pandas")
4650
else:
47-
PANDAS_INSTALLED_VERSION = pkg_resources.parse_version("0.0.0")
48-
49-
PANDAS_INT64_VERSION = pkg_resources.parse_version("1.0.0")
51+
PANDAS_INSTALLED_VERSION = "0.0.0"
5052

5153

5254
class MissingDataError(Exception):
@@ -310,10 +312,7 @@ def test_load_table_from_dataframe_w_automatic_schema(bigquery_client, dataset_i
310312
]
311313

312314

313-
@pytest.mark.skipif(
314-
PANDAS_INSTALLED_VERSION < PANDAS_INT64_VERSION,
315-
reason="Only `pandas version >=1.0.0` is supported",
316-
)
315+
@pytest.mark.skipif(pandas is None, reason="Requires `pandas`")
317316
def test_load_table_from_dataframe_w_nullable_int64_datatype(
318317
bigquery_client, dataset_id
319318
):
@@ -342,7 +341,7 @@ def test_load_table_from_dataframe_w_nullable_int64_datatype(
342341

343342

344343
@pytest.mark.skipif(
345-
PANDAS_INSTALLED_VERSION < PANDAS_INT64_VERSION,
344+
PANDAS_INSTALLED_VERSION[0:2].startswith("0."),
346345
reason="Only `pandas version >=1.0.0` is supported",
347346
)
348347
def test_load_table_from_dataframe_w_nullable_int64_datatype_automatic_schema(
@@ -1043,9 +1042,7 @@ def test_list_rows_max_results_w_bqstorage(bigquery_client):
10431042
assert len(dataframe.index) == 100
10441043

10451044

1046-
@pytest.mark.skipif(
1047-
PANDAS_INSTALLED_VERSION >= pkg_resources.parse_version("2.0.0"), reason=""
1048-
)
1045+
@pytest.mark.skipif(PANDAS_INSTALLED_VERSION[0:2] not in ["0.", "1."], reason="")
10491046
@pytest.mark.parametrize(
10501047
("max_results",),
10511048
(
Collapse file

‎tests/unit/job/test_query_pandas.py‎

Copy file name to clipboardExpand all lines: tests/unit/job/test_query_pandas.py
+8-6Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@
1717
import json
1818

1919
import mock
20-
import pkg_resources
2120
import pytest
2221

2322

@@ -45,14 +44,19 @@
4544
except (ImportError, AttributeError): # pragma: NO COVER
4645
tqdm = None
4746

47+
try:
48+
import importlib.metadata as metadata
49+
except ImportError:
50+
import importlib_metadata as metadata
51+
4852
from ..helpers import make_connection
4953
from .helpers import _make_client
5054
from .helpers import _make_job_resource
5155

5256
if pandas is not None:
53-
PANDAS_INSTALLED_VERSION = pkg_resources.get_distribution("pandas").parsed_version
57+
PANDAS_INSTALLED_VERSION = metadata.version("pandas")
5458
else:
55-
PANDAS_INSTALLED_VERSION = pkg_resources.parse_version("0.0.0")
59+
PANDAS_INSTALLED_VERSION = "0.0.0"
5660

5761
pandas = pytest.importorskip("pandas")
5862

@@ -656,9 +660,7 @@ def test_to_dataframe_bqstorage_no_pyarrow_compression():
656660
)
657661

658662

659-
@pytest.mark.skipif(
660-
PANDAS_INSTALLED_VERSION >= pkg_resources.parse_version("2.0.0"), reason=""
661-
)
663+
@pytest.mark.skipif(PANDAS_INSTALLED_VERSION[0:2] not in ["0.", "1."], reason="")
662664
@pytest.mark.skipif(pyarrow is None, reason="Requires `pyarrow`")
663665
def test_to_dataframe_column_dtypes():
664666
from google.cloud.bigquery.job import QueryJob as target_class
Collapse file

‎tests/unit/test__pandas_helpers.py‎

Copy file name to clipboardExpand all lines: tests/unit/test__pandas_helpers.py
+10-22Lines changed: 10 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,11 @@
1919
import operator
2020
import queue
2121
import warnings
22-
import pkg_resources
22+
23+
try:
24+
import importlib.metadata as metadata
25+
except ImportError:
26+
import importlib_metadata as metadata
2327

2428
import mock
2529

@@ -57,13 +61,10 @@
5761

5862
bigquery_storage = _versions_helpers.BQ_STORAGE_VERSIONS.try_import()
5963

60-
PANDAS_MINIUM_VERSION = pkg_resources.parse_version("1.0.0")
61-
6264
if pandas is not None:
63-
PANDAS_INSTALLED_VERSION = pkg_resources.get_distribution("pandas").parsed_version
65+
PANDAS_INSTALLED_VERSION = metadata.version("pandas")
6466
else:
65-
# Set to less than MIN version.
66-
PANDAS_INSTALLED_VERSION = pkg_resources.parse_version("0.0.0")
67+
PANDAS_INSTALLED_VERSION = "0.0.0"
6768

6869

6970
skip_if_no_bignumeric = pytest.mark.skipif(
@@ -542,9 +543,7 @@ def test_bq_to_arrow_array_w_nullable_scalars(module_under_test, bq_type, rows):
542543
],
543544
)
544545
@pytest.mark.skipif(pandas is None, reason="Requires `pandas`")
545-
@pytest.mark.skipif(
546-
PANDAS_INSTALLED_VERSION >= pkg_resources.parse_version("2.0.0"), reason=""
547-
)
546+
@pytest.mark.skipif(PANDAS_INSTALLED_VERSION[0:2] not in ["0.", "1."], reason="")
548547
@pytest.mark.skipif(isinstance(pyarrow, mock.Mock), reason="Requires `pyarrow`")
549548
def test_bq_to_arrow_array_w_pandas_timestamp(module_under_test, bq_type, rows):
550549
rows = [pandas.Timestamp(row) for row in rows]
@@ -806,10 +805,7 @@ def test_list_columns_and_indexes_with_named_index_same_as_column_name(
806805
assert columns_and_indexes == expected
807806

808807

809-
@pytest.mark.skipif(
810-
pandas is None or PANDAS_INSTALLED_VERSION < PANDAS_MINIUM_VERSION,
811-
reason="Requires `pandas version >= 1.0.0` which introduces pandas.NA",
812-
)
808+
@pytest.mark.skipif(pandas is None, reason="Requires `pandas`")
813809
def test_dataframe_to_json_generator(module_under_test):
814810
utcnow = datetime.datetime.utcnow()
815811
df_data = collections.OrderedDict(
@@ -837,16 +833,8 @@ def test_dataframe_to_json_generator(module_under_test):
837833
assert list(rows) == expected
838834

839835

836+
@pytest.mark.skipif(pandas is None, reason="Requires `pandas`")
840837
def test_dataframe_to_json_generator_repeated_field(module_under_test):
841-
pytest.importorskip(
842-
"pandas",
843-
minversion=str(PANDAS_MINIUM_VERSION),
844-
reason=(
845-
f"Requires `pandas version >= {PANDAS_MINIUM_VERSION}` "
846-
"which introduces pandas.NA"
847-
),
848-
)
849-
850838
df_data = [
851839
collections.OrderedDict(
852840
[("repeated_col", [pandas.NA, 2, None, 4]), ("not_repeated_col", "first")]
Collapse file

‎tests/unit/test_client.py‎

Copy file name to clipboardExpand all lines: tests/unit/test_client.py
+9-14Lines changed: 9 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,11 @@
3030
import requests
3131
import packaging
3232
import pytest
33-
import pkg_resources
33+
34+
try:
35+
import importlib.metadata as metadata
36+
except ImportError:
37+
import importlib_metadata as metadata
3438

3539
try:
3640
import pandas
@@ -76,13 +80,10 @@
7680
from test_utils.imports import maybe_fail_import
7781
from tests.unit.helpers import make_connection
7882

79-
PANDAS_MINIUM_VERSION = pkg_resources.parse_version("1.0.0")
80-
8183
if pandas is not None:
82-
PANDAS_INSTALLED_VERSION = pkg_resources.get_distribution("pandas").parsed_version
84+
PANDAS_INSTALLED_VERSION = metadata.version("pandas")
8385
else:
84-
# Set to less than MIN version.
85-
PANDAS_INSTALLED_VERSION = pkg_resources.parse_version("0.0.0")
86+
PANDAS_INSTALLED_VERSION = "0.0.0"
8687

8788

8889
def _make_credentials():
@@ -8145,10 +8146,7 @@ def test_load_table_from_dataframe_unknown_table(self):
81458146
timeout=DEFAULT_TIMEOUT,
81468147
)
81478148

8148-
@unittest.skipIf(
8149-
pandas is None or PANDAS_INSTALLED_VERSION < PANDAS_MINIUM_VERSION,
8150-
"Only `pandas version >=1.0.0` supported",
8151-
)
8149+
@unittest.skipIf(pandas is None, "Requires `pandas`")
81528150
@unittest.skipIf(pyarrow is None, "Requires `pyarrow`")
81538151
def test_load_table_from_dataframe_w_nullable_int64_datatype(self):
81548152
from google.cloud.bigquery.client import _DEFAULT_NUM_RETRIES
@@ -8193,10 +8191,7 @@ def test_load_table_from_dataframe_w_nullable_int64_datatype(self):
81938191
SchemaField("x", "INT64", "NULLABLE", None),
81948192
)
81958193

8196-
@unittest.skipIf(
8197-
pandas is None or PANDAS_INSTALLED_VERSION < PANDAS_MINIUM_VERSION,
8198-
"Only `pandas version >=1.0.0` supported",
8199-
)
8194+
@unittest.skipIf(pandas is None, "Requires `pandas`")
82008195
# @unittest.skipIf(pyarrow is None, "Requires `pyarrow`")
82018196
def test_load_table_from_dataframe_w_nullable_int64_datatype_automatic_schema(self):
82028197
from google.cloud.bigquery.client import _DEFAULT_NUM_RETRIES
Collapse file

‎tests/unit/test_packaging.py‎

Copy file name to clipboard
+37Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Copyright 2023 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import os
16+
import subprocess
17+
import sys
18+
19+
20+
def test_namespace_package_compat(tmp_path):
21+
# The ``google`` namespace package should not be masked
22+
# by the presence of ``google-cloud-bigquery``.
23+
google = tmp_path / "google"
24+
google.mkdir()
25+
google.joinpath("othermod.py").write_text("")
26+
env = dict(os.environ, PYTHONPATH=str(tmp_path))
27+
cmd = [sys.executable, "-m", "google.othermod"]
28+
subprocess.check_call(cmd, env=env)
29+
30+
# The ``google.cloud`` namespace package should not be masked
31+
# by the presence of ``google-cloud-bigquery``.
32+
google_cloud = tmp_path / "google" / "cloud"
33+
google_cloud.mkdir()
34+
google_cloud.joinpath("othermod.py").write_text("")
35+
env = dict(os.environ, PYTHONPATH=str(tmp_path))
36+
cmd = [sys.executable, "-m", "google.cloud.othermod"]
37+
subprocess.check_call(cmd, env=env)

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.