Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Support Array type in log table (Arrow row format)#1987

Merged
wuchong merged 2 commits intoapache:mainapache/fluss:mainfrom
XuQianJin-Stars:feature/issue-1972-array-in-log-tableXuQianJin-Stars/fluss:feature/issue-1972-array-in-log-tableCopy head branch name to clipboard
Nov 24, 2025
Merged

Support Array type in log table (Arrow row format)#1987
wuchong merged 2 commits intoapache:mainapache/fluss:mainfrom
XuQianJin-Stars:feature/issue-1972-array-in-log-tableXuQianJin-Stars/fluss:feature/issue-1972-array-in-log-tableCopy head branch name to clipboard

Conversation

@XuQianJin-Stars
Copy link
Copy Markdown
Contributor

@XuQianJin-Stars XuQianJin-Stars commented Nov 16, 2025

Purpose

Linked issue: close #1972 #1975

This PR removes the getMap() and getRow() methods from DataGetters interface and all implementing classes, as these methods are not yet fully supported and will be added later in Issue #1973 (Map support) and Issue #1974 (Row support). Additionally, this PR improves test coverage for array-related classes to meet the 70% coverage requirement.

Brief change log

API Changes:

  • Removed getMap(int pos) method from DataGetters interface and all implementations (9 classes: BinaryArray, GenericArray, GenericRow, IndexedRow, AlignedRow, ProjectedRow, ColumnarRow, ColumnarArray, CompactedRow)
  • Removed getRow(int pos, int numFields) method from DataGetters interface and all implementations
  • Updated InternalArray.createElementGetter() to throw IllegalArgumentException for MAP and ROW types
  • Updated InternalRow.createFieldGetter() to throw IllegalArgumentException for MAP and ROW types
  • Removed getMap() and getRow() helper methods from VectorizedColumnBatch
  • Added TODO comments indicating these features will be added in Issue Support Map type in log table (Arrow row format) #1973 and Support NestedRow type in log table (Arrow row format) #1974

Test Improvements:

  • InternalArraySerializer: Added 3 new tests (23 total tests, coverage: 68% → 70%+)
  • BinaryArray: Added 3 new tests (64 total tests, coverage: 57% → 60%+)
  • BinarySegmentUtils: Added 12 new tests (31 total tests, coverage: 45% → 70%+)
  • Removed/commented out tests that depended on getMap() and getRow() methods:
    • InternalArrayTest.testElementGetterForMap() and testElementGetterForRow()
    • GenericArrayTest.testNestedRow()
  • Updated InternalArrayAssert to handle MAP type as unsupported

Code Cleanup:

  • Removed unused imports: InternalMap, InternalRow, DataTypeRoot, DataField, DataTypeChecks
  • Removed unused private method getDataTypes() from IndexedRow

Tests

Unit Tests:

  • All existing tests pass (1244 tests)
  • New tests added:
    • InternalArraySerializerTest: testToBinaryArrayFromGenericArrayWithBytes, testEqualsWithNull, testCopyGenericArrayAllNulls
    • BinaryArrayTest: testGetSizeInBytes, testCopyWithMultipleTypes, testHashCodeConsistency
    • BinarySegmentUtilsTest: testSetBoolean/Byte/Short/Int/Float/Double, testBitSetAndUnSet, testReadBinary, testCopyToBytes
  • Coverage verification: mvn clean verify -pl fluss-test-coverage passes with all classes meeting 70% coverage requirement

API and Format

API Changes (Breaking):

No format changes.

Documentation

@XuQianJin-Stars XuQianJin-Stars force-pushed the feature/issue-1972-array-in-log-table branch 3 times, most recently from 1cb52e9 to d02b31f Compare November 17, 2025 00:55
@wuchong wuchong force-pushed the feature/issue-1972-array-in-log-table branch 3 times, most recently from 30029fa to e588330 Compare November 23, 2025 09:53
@wuchong wuchong force-pushed the feature/issue-1972-array-in-log-table branch from 29d60c8 to 3620267 Compare November 23, 2025 10:34
@wuchong
Copy link
Copy Markdown
Member

wuchong commented Nov 23, 2025

@XuQianJin-Stars @platinumhamburg I have reviewed and polished the pull request. Please take a look. It mainly reverts some changes that not necessary and improved the implementation.

@wuchong wuchong merged commit 05a5db9 into apache:main Nov 24, 2025
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Array type in log table (Arrow row format)

2 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.