Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

feat: allow setting table labels in to_gbq #941

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
Sep 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
6257ad7
first commit:
Aug 5, 2024
60a542a
Merge branch 'googleapis:main' into main
mattyopl Aug 5, 2024
f5e8432
Revert "first commit:"
Aug 6, 2024
2f11e54
Merge branch 'googleapis:main' into dev
mattyopl Aug 6, 2024
c6acd75
Merge branch 'googleapis:main' into dev
mattyopl Aug 12, 2024
b506c4c
chore: allow setting table labels in `to_gbq`
Aug 30, 2024
db31624
Merge branch 'googleapis:main' into dev
mattyopl Aug 30, 2024
0b09216
chore: allow setting table labels in `to_gbq`
Aug 30, 2024
de4e416
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Aug 30, 2024
44775d5
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Aug 30, 2024
c30c52a
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Aug 30, 2024
35b891e
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Aug 30, 2024
81bb23b
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Aug 30, 2024
1a291ee
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Aug 30, 2024
62e2627
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Aug 30, 2024
2d8b220
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Aug 30, 2024
8982149
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Aug 30, 2024
dfd1c3a
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Aug 30, 2024
efd48b6
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Aug 30, 2024
9084327
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Aug 30, 2024
1c09f7d
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Aug 30, 2024
9ae5546
Merge branch 'main' into dev
mattyopl Sep 3, 2024
90618f6
Merge branch 'main' into dev
mattyopl Sep 3, 2024
9f0dc25
Merge branch 'dev' of github.com:mattyopl/python-bigquery-dataframes …
Sep 4, 2024
84342f1
Merge branch 'main' into dev
mattyopl Sep 4, 2024
9f08c6d
Merge branch 'main' into dev
chelsea-lin Sep 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions 14 bigframes/dataframe.py
Original file line number Diff line number Diff line change
Expand Up @@ -3027,6 +3027,7 @@ def to_gbq(
index: bool = True,
ordering_id: Optional[str] = None,
clustering_columns: Union[pandas.Index, Iterable[typing.Hashable]] = (),
labels: dict[str, str] = {},
mattyopl marked this conversation as resolved.
Show resolved Hide resolved
) -> str:
temp_table_ref = None

Expand Down Expand Up @@ -3081,9 +3082,11 @@ def to_gbq(
export_array, id_overrides = self._prepare_export(
index=index and self._has_index, ordering_id=ordering_id
)
destination = bigquery.table.TableReference.from_string(
destination_table,
default_project=default_project,
destination: bigquery.table.TableReference = (
bigquery.table.TableReference.from_string(
destination_table,
default_project=default_project,
)
)
_, query_job = self._session._export(
export_array,
Expand All @@ -3106,6 +3109,11 @@ def to_gbq(
+ constants.DEFAULT_EXPIRATION,
)

if len(labels) != 0:
mattyopl marked this conversation as resolved.
Show resolved Hide resolved
table = bigquery.Table(result_table)
table.labels = labels
self._session.bqclient.update_table(table, ["labels"])

return destination_table

def to_numpy(
Expand Down
11 changes: 11 additions & 0 deletions 11 tests/system/small/test_dataframe.py
Original file line number Diff line number Diff line change
Expand Up @@ -4657,6 +4657,17 @@ def test_to_gbq_and_create_dataset(session, scalars_df_index, dataset_id_not_cre
assert not loaded_scalars_df_index.empty


def test_to_gbq_table_labels(scalars_df_index):
destination_table = "bigframes-dev.bigframes_tests_sys.table_labels"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally the test should not be hard coupled with a project/dataset. Do we have to use a fixed table name? I think the test would still serve its purpose if we did to_gbq without an explicit table arg and asserted labels on the returned anonymous table.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done in #968!

result_table = scalars_df_index.to_gbq(
destination_table, labels={"test": "labels"}, if_exists="replace"
)
client = scalars_df_index._session.bqclient
table = client.get_table(result_table)
assert table.labels
assert table.labels["test"] == "labels"


@pytest.mark.parametrize(
("col_names", "ignore_index"),
[
Expand Down
4 changes: 4 additions & 0 deletions 4 third_party/bigframes_vendored/pandas/core/frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -390,6 +390,7 @@ def to_gbq(
index: bool = True,
ordering_id: Optional[str] = None,
clustering_columns: Union[pd.Index, Iterable[Hashable]] = (),
labels: dict[str, str] = {},
) -> str:
"""Write a DataFrame to a BigQuery table.

Expand Down Expand Up @@ -467,6 +468,9 @@ def to_gbq(
clustering order within the Index/DataFrame columns follows the order
specified in `clustering_columns`.

labels (dict[str, str], default None):
Specifies table labels within BigQuery
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, (but since this goes in the public doc) should end with the period .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done in #968!


Returns:
str:
The fully-qualified ID for the written table, in the form
Expand Down
Morty Proxy This is a proxified and sanitized view of the page, visit original site.