Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

BUG: AttributeError: 'BlockManager' object has no attribute 'is_mixed_type' when trying to jsonify dataframe. #39837

Copy link
Copy link
Closed
@yuvalmarciano

Description

@yuvalmarciano
Issue body actions
  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Code Sample, a copy-pastable example

csv_reader = pandas.read_csv(
             csv_file, iterator=True, chunksize=2500, sep=',', encoding=None, error_bad_lines=False
     )
for i, chunk in enumerate(csv_reader):
     rows = chunk.to_json(orient="records", lines=True)

Problem description

on pypy 7.3.3 - python 3.7.9
to_json execution raises an exception:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/opt/pypy/site-packages/pandas/core/generic.py", line 2478, in to_json
    storage_options=storage_options,
  File "/opt/pypy/site-packages/pandas/io/json/_json.py", line 94, in to_json
    indent=indent,
  File "/opt/pypy/site-packages/pandas/io/json/_json.py", line 155, in write
    indent=self.indent,
AttributeError: 'BlockManager' object has no attribute 'is_mixed_type'

I suspect this PR: https://github.com/pandas-dev/pandas/pull/36873/files
@jbrockmendel do you mind explaining why the BlockManager no longer has the is_mixed_type property?

Expected Output

When I try to manually assign is_mixed_type to the chunk._mgr object, as dropped in the aforementioned PR (a BlockManager) it seems to be working fine:

for i, chunk in enumerate(csv_reader):
     chunk._mgr._consolidate_inplace()
     chunk._mgr.is_mixed_type = len(chunk._mgr.blocks) > 1
     print(chunk.to_json(orient="records", lines=True))

{"@timestamp":10000000,"test_1":"something","test_2":"something2"}

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 7d32926
python : 3.7.9.final.0
python-bits : 64
OS : Linux
OS-release : 4.19.121-linuxkit
Version : #1 SMP Tue Dec 1 17:50:32 UTC 2020
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : C.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.2.2
numpy : 1.20.1
pytz : 2021.1
dateutil : 2.8.1
pip : 20.3.4
setuptools : 53.0.0
Cython : None
pytest : 6.2.2
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : 2.8.6 (dt dec pq3 ext lo64)
jinja2 : 2.11.3
IPython : 7.20.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : 1.3.23
tables : None
tabulate : 0.8.7
xarray : None
xlrd : 1.2.0
xlwt : None
numba : None

lior-l, hekmati, tincho9, rkissos, yogevyuval and 1 more

Metadata

Metadata

Assignees

No one assigned

    Labels

    IO JSONread_json, to_json, json_normalizeread_json, to_json, json_normalizeRegressionFunctionality that used to work in a prior pandas versionFunctionality that used to work in a prior pandas version

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Morty Proxy This is a proxified and sanitized view of the page, visit original site.