Description
Pandas version checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
df = pd.DataFrame(
{"date": pd.date_range("2023-01-01", periods=5, unit="s"), "values": 1}
)
# datetime64[s]
print(df["date"].dtype)
grouped = df.groupby("date", as_index=False).sum()
# datetime64[ns]
print(grouped["date"].dtype)
concatted = pd.concat([df, grouped])["date"]
# object
print(concatted.dtype)Issue Description
I stumbled upon this behavior when test-driving 2.0rc0 with my code base.
Some data frames now contain date columns of type datetime64[s]. Did not investigate yet why. The problem occurs when working with these columns. After a groupby, the type changes to datetime64[ns].
When now concatting datetime64[s] to datetime64[ns] things get even worse and I have an object column.
Expected Behavior
As I started with a datetime column, I'd expect the datetime column dtype not to change magically. If a later pd.merge wouldn't have complained, it could have been hard to track down what was going on.
Installed Versions
INSTALLED VERSIONS
commit : 1a2e300
python : 3.11.0.final.0
python-bits : 64
OS : Darwin
OS-release : 22.1.0
Version : Darwin Kernel Version 22.1.0: Sun Oct 9 20:14:54 PDT 2022; root:xnu-8792.41.9~2/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : None
LOCALE : None.UTF-8
pandas : 2.0.0rc0
numpy : 1.24.2
pytz : 2022.7.1
dateutil : 2.8.2
setuptools : 67.4.0
pip : 23.0.1
Cython : None
pytest : 7.2.1
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.9.2
html5lib : 1.1
pymysql : None
psycopg2 : 2.9.5
jinja2 : 3.1.2
IPython : 8.11.0
pandas_datareader: None
bs4 : 4.11.2
bottleneck : 1.3.7
brotli : None
fastparquet : None
fsspec : 2023.1.0
gcsfs : 2023.1.0
matplotlib : 3.7.0
numba : None
numexpr : 2.8.4
odfpy : None
openpyxl : 3.1.0
pandas_gbq : None
pyarrow : 11.0.0
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.10.1
snappy : None
sqlalchemy : 1.4.46
tables : None
tabulate : 0.9.0
xarray : None
xlrd : None
zstandard : None
tzdata : None
qtpy : None
pyqt5 : None