Turn ObjectStoreLocationProvider off by default
#1722
Conversation
Fokko
left a comment
There was a problem hiding this comment.
Thanks for fixing this @kevinjqliu.
I think it would be good to add this to 0.9.0
|
reverts #1509, cc @smaheshwar-pltr |
|
Thanks for the ping @kevinjqliu. (Feel free to respond later) Curious what the problem here was. I'd have thought that old tables would still be readable by new PyIceberg and old PyIceberg would still be able to read tables created by the new one. Is changing locations just too dramatic for this release? @Fokko's #1452 (comment) that prompted this still seems reasonable (maybe longer-term), or is it not? |
My main concern was around the default behavior change. For example, if i had a pipeline running with 0.8.1 and now i upgrade to 0.9.0. The data file location is suddenly changed to use the hashed path. We can add it to the release notes, but i think changing the default behavior will surprise some users. |
|
Thanks @kevinjqliu for the explanation. |
|
:) Sorry for the back and forth! |
Closes apache#1721 Otherwise, there's a default behavior change in new 0.9.0 release. Previous versions will write to `data/`, new version will write to `data/<hash>/`
Update the docs to reflect the simple location provider as the default location provider. related to apache#1721, and apache#1722.
Closes #1721
Otherwise, there's a default behavior change in new 0.9.0 release. Previous versions will write to
data/, new version will write todata/<hash>/