[GLUTEN-12597][CORE] Migrate Substrait extension referencing from URI to URN#12604
Open
nielspardon wants to merge 1 commit into
apache:mainapache/gluten:mainfrom
nielspardon:feat/substrait-0.98-inc1-uri-to-urnnielspardon/gluten:feat/substrait-0.98-inc1-uri-to-urnCopy head branch name to clipboard
Open
[GLUTEN-12597][CORE] Migrate Substrait extension referencing from URI to URN#12604nielspardon wants to merge 1 commit intoapache:mainapache/gluten:mainfrom nielspardon:feat/substrait-0.98-inc1-uri-to-urnnielspardon/gluten:feat/substrait-0.98-inc1-uri-to-urnCopy head branch name to clipboard
nielspardon wants to merge 1 commit into
apache:mainapache/gluten:mainfrom
nielspardon:feat/substrait-0.98-inc1-uri-to-urnnielspardon/gluten:feat/substrait-0.98-inc1-uri-to-urnCopy head branch name to clipboard
Conversation
|
Run Gluten Clickhouse CI on x86 |
… to URN Adopt the 0.98 URN-based simple-extension referencing model (substrait-io/substrait#971): - extensions.proto: SimpleExtensionURI -> SimpleExtensionURN; extension_uri_anchor -> extension_urn_anchor; uri -> urn; and in ExtensionType/ExtensionTypeVariation/ExtensionFunction, extension_uri_reference (field 1) is reserved and replaced by extension_urn_reference (field 4). - plan.proto / extended_expression.proto: extension_uris (field 1) is reserved, replaced by extension_urns = 8 (repeated SimpleExtensionURN). Velox producer (SubstraitExtensionCollector, VeloxToSubstraitPlan) now emits extension_urns / extension_urn_reference. Gluten still maps every function to one catch-all anchor and resolves by name, so the emitted URN is a single placeholder in the required extension:<OWNER>:<ID> format (extension:org.apache.gluten:functions) rather than an empty string; consuming the upstream io.substrait function extensions is a follow-up. Gluten's JVM producer never emitted extension_uris, so it needs no changes beyond recompiling against the regenerated classes. Test fixtures updated to URN field names and proper spec URN values (extension:io.substrait:<file>). AdvancedExtension.optimization (also repeated in 0.98) and the additive Plan fields (parameter_bindings/type_aliases/execution_behavior) are handled in follow-up increments. Part of apache#12597. Validated locally: gluten-substrait JVM build + Velox native build (libgluten/libvelox).
nielspardon
force-pushed
the
feat/substrait-0.98-inc1-uri-to-urn
branch
from
July 23, 2026 14:25
4e261ff to
9f0fa46
Compare
|
Run Gluten Clickhouse CI on x86 |
zhouyuan
approved these changes
Jul 24, 2026
zhouyuan
left a comment
Member
There was a problem hiding this comment.
LGTM, thanks for the fix @nielspardon !
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
Second increment of the Substrait 0.98 proto rebase (#12597). Stacked on #12598 (approved, not yet merged) — until #12598 lands, this PR's diff includes its commit; the change owned by this PR is the top commit (URI→URN). I'll rebase onto
mainonce #12598 merges, which reduces the diff to just this increment. (The two increments touch disjoint files.)Adopts the 0.98 URN-based simple-extension referencing model (substrait-io/substrait#971), replacing URI-based references with URNs of the form
extension:<OWNER>:<ID>:extensions.proto:SimpleExtensionURI→SimpleExtensionURN;extension_uri_anchor→extension_urn_anchor;uri→urn; and inExtensionType/ExtensionTypeVariation/ExtensionFunction,extension_uri_reference(field 1) is reserved and replaced byextension_urn_reference(field 4).plan.proto/extended_expression.proto:extension_uris(field 1) reserved, replaced byextension_urns = 8(repeated SimpleExtensionURN).SubstraitExtensionCollector,VeloxToSubstraitPlan): now emitextension_urns/extension_urn_reference. Gluten maps all functions to a single catch-all anchor and resolves them by name, so it emits one valid-format placeholder URNextension:org.apache.gluten:functionsinstead of an empty string; consuming the real upstreamio.substraitfunction-extension URNs is a follow-up. Gluten's JVM producer never emittedextension_uris, so it needs no changes beyond recompiling against the regenerated classes.extension:io.substrait:<file>).AdvancedExtension.optimization(alsorepeatedin 0.98) and the additivePlanfields (parameter_bindings/type_aliases/execution_behavior) are handled in follow-up increments.How was this patch tested?
No behavioral change — this is a referencing-mechanism migration. Verified locally: the vendored proto compiles (
protoc); thegluten-substraitJVM build succeeds (regenerating theSimpleExtensionURNclasses); and the Velox native library builds (libgluten/libvelox, recompilingSubstraitExtensionCollector/VeloxToSubstraitPlan/SubstraitParseragainst the regenerated proto). The ClickHouse backend has noextension_uri*references and is unaffected. Existing Velox conversion tests exercise the updated fixtures.Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)
🤖 Generated with AI