Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Fail to unpickle PickleType #654

Copy link
Copy link
Closed
@baely

Description

@baely
Issue body actions

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Is this a client library issue or a product issue? We will only be able to assist with issues that pertain to the behaviors of this library. If the issue you're experiencing is due to the behavior of the product itself, please visit the Support page to reach the most relevant engineers.

If the support paths suggested above still do not result in a resolution, please provide the following details.

Environment details

  • Programming language: Python 3.13
  • OS: MacOS 15
  • Language runtime version:
  • Package version: 1.10.0

Steps to reproduce

  1. Define class with column of type sqlalchemy PickleType
  2. Add instance of class to Spanner
  3. Attempt to read the row

Sample:

import os

from sqlalchemy import Column, Integer, String, PickleType, create_engine
from sqlalchemy.orm import DeclarativeBase, Session

class Base(DeclarativeBase):
    pass

class UserPreferences(Base):
    __tablename__ = 'user_preferences'
    
    user_id = Column(Integer, primary_key=True)
    username = Column(String(50), nullable=False)
    preferences = Column(PickleType, nullable=True)
    created_at = Column(String(30), nullable=False)


os.environ["SPANNER_EMULATOR_HOST"] = "localhost:9010"
engine = create_engine("spanner+spanner:///projects/test-project/instances/test-instance/databases/test-database")

with Session(engine) as session:
    new_user = UserPreferences(
        user_id=1,
        username='test_user',
        preferences={
            'setting': 'true'
        },
        created_at="2025-05-04T00:00:00.000000"
    )

    session.add(new_user)
    session.commit()

    user = session.query(UserPreferences).filter_by(user_id=1).first()

Will raise an exception on the last line: _pickle.UnpicklingError: pickle data was truncated

The data is being base64 encoded on the way out from Spanner where SQLAlchemy is expecting raw bytes for pickled data. The data is stored as raw bytes in Spanner but being encoded on the way out.

Image Image

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

Metadata

Metadata

Assignees

Labels

api: spannerIssues related to the googleapis/python-spanner-sqlalchemy API.Issues related to the googleapis/python-spanner-sqlalchemy API.priority: p2Moderately-important priority. Fix may not be included in next release.Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    Morty Proxy This is a proxified and sanitized view of the page, visit original site.