Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

#49 Implement Python native read with PyArrow#50

Merged
yuzelin merged 3 commits into
apache:mainapache/paimon-python:mainfrom
chenghuichen:pyarrow-part1chenghuichen/paimon-python:pyarrow-part1Copy head branch name to clipboard
May 22, 2025
Merged

#49 Implement Python native read with PyArrow#50
yuzelin merged 3 commits into
apache:mainapache/paimon-python:mainfrom
chenghuichen:pyarrow-part1chenghuichen/paimon-python:pyarrow-part1Copy head branch name to clipboard

Conversation

@chenghuichen
Copy link
Copy Markdown
Contributor

@chenghuichen chenghuichen commented Apr 6, 2025

Purpose

Linked issue: close #49

This PR implements the first phase of Python native operations using PyArrow, focusing on read operations only. This is part of the larger effort to improve paimon-python performance as discussed in #49.

Tests

14 cases in pypaimon/pynative/tests/test_pynative_reader.py

API and Format

no

Documentation

@chenghuichen chenghuichen changed the title Update .gitignore [WIP] Implement Python native read with PyArrow Apr 7, 2025
@chenghuichen chenghuichen changed the title [WIP] Implement Python native read with PyArrow #49 [WIP] Implement Python native read with PyArrow Apr 7, 2025
@chenghuichen chenghuichen changed the title #49 [WIP] Implement Python native read with PyArrow #49 Implement Python native read with PyArrow Apr 18, 2025
Copy link
Copy Markdown
Contributor

@yuzelin yuzelin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@yuzelin yuzelin merged commit b4dd596 into apache:main May 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Performance Improvement: Replace Py4J-based Implementation with Native PyArrow

2 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.