Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

pyarrow.lib.ArrowTypeError: Expected bytes, got a 'dict' object #452

Copy link
Copy link
Closed
@ivanpugachtd

Description

@ivanpugachtd
Issue body actions

Hi!
there is a problem when trying to load using pandas-gbq which using pyarrow a column of the list (array) or dictionary (json) type into the table, while the GBQ documentation says that structure types such as array or json are supported,

df = pd.DataFrame(
                {
                    "my_string": ["a", "b", "c"],
                    "my_int64": [1, 2, 3],
                    "my_float64": [4.0, 5.0, 6.0],
                    "my_bool1": [True, False, True],
                    "my_bool2": [False, True, False],
                    "my_struct": [{"test":"str1"},{"test":"str2"},{"test":"str3"}],
                }
            )
pandas_gbq.to_gbq(df, **gbq_params)

as a result, a stacktrace error occurs:

  • in bq_to_arrow_array
  • return pyarrow.Array.from_pandas(series, type=arrow_type)
  • File "pyarrow/array.pxi", line 913, in pyarrow.lib.Array.from_pandas
  • File "pyarrow/array.pxi", line 311, in pyarrow.lib.array
  • File "pyarrow/array.pxi", line 83, in pyarrow.lib._ndarray_to_array
  • File "pyarrow/error.pxi", line 122, in pyarrow.lib.check_status
  • pyarrow.lib.ArrowTypeError: Expected bytes, got a 'dict' object

Can anyone help with it please?

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the googleapis/python-bigquery-pandas API.Issues related to the googleapis/python-bigquery-pandas API.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.‘Nice-to-have’ improvement, new feature or different behavior or design.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    Morty Proxy This is a proxified and sanitized view of the page, visit original site.