Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
This repository was archived by the owner on May 7, 2026. It is now read-only.
This repository was archived by the owner on May 7, 2026. It is now read-only.

DataFrame.to_gbq with if_exists="replace" and clustering_columns can swap data between columns if order is different #2502

Copy link
Copy link

Description

@jbandoro
Issue body actions

Environment details

Python: 3.12.10 (main, Apr  9 2025, 03:49:38) [Clang 20.1.0 ]
bigframes==2.37.0
google-cloud-bigquery==3.40.1
pandas==3.0.1
pyarrow==23.0.1

Steps to reproduce

  1. Create a table with Dataframe.to_gbq with clustering_columns
  2. Reorder the clustering column and another column that has the same type
  3. Write the updated dataframe to the same table with if_exists="replace"
  4. Inspect the table to see columns with different data than in dataframe

Code example

import bigframes.pandas as bpd
import pandas as pd

TABLE_ID = "my_project.my_dataset.test_table"

df = pd.DataFrame(
    {"a": [10, 1, 2], "b": ["Justin", "Alex", "Jim"], "c": ["cat", "dog", "cat"]}
)
bdf = bpd.DataFrame(df)

# First time writing, all good
bdf.to_gbq(TABLE_ID, clustering_columns=["b"])

# Reorder the clustered column b and column c
bdf = bdf[["a", "c", "b"]]

# When replacing and writing, the b and c columns are swapped
bdf.to_gbq(TABLE_ID, if_exists="replace", clustering_columns=["b"])

While the dataframe looks good, the updated table now has columns b/c data swapped:

Image Image

This doesn't happen if clustering_columns is not used in my testing.

Reactions are currently unavailable

Metadata

Metadata

Labels

api: bigqueryIssues related to the googleapis/python-bigquery-dataframes API.Issues related to the googleapis/python-bigquery-dataframes API.

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    Morty Proxy This is a proxified and sanitized view of the page, visit original site.