feat(core): add VECTOR datatype support with Oracle vector integration#18227
Open
prajalg wants to merge 52 commits into
Open
feat(core): add VECTOR datatype support with Oracle vector integration#18227prajalg wants to merge 52 commits into
prajalg wants to merge 52 commits into
Conversation
… vector functions
…or distance function
…etc. validate using to only allow hnsw or ivf allowlist distance metrics before emitting WITH DISTANCE require numeric, finite, positive values for accuracy and vector index parameters quote dropConstraintQuery constraint names
…e-dataaccess-sequelize/sequelize-oracle into vector_support_v7
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThis PR adds VECTOR datatype support across core and Oracle: core type contracts and export, Sequelize vector helper methods, Oracle-specific datatype overrides and SQL formatting, CREATE VECTOR INDEX generation, show-indexes projection updates, and comprehensive unit and integration tests. ChangesOracle Vector Data Type Support
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (2)
packages/oracle/src/_internal/data-types-overrides.ts (1)
549-565: 💤 Low valueRedundant undefined check.
Number.isInteger(numDimensions)already returnsfalseforundefined, making thenumDimensions !== undefinedcheck redundant.Simplify the condition
return ( isVectorComponent(values) && isVectorComponent(indices) && Number.isInteger(numDimensions) && - numDimensions !== undefined && numDimensions > 0 && values.length === indices.length );🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/oracle/src/_internal/data-types-overrides.ts` around lines 549 - 565, In isOracleSparseVectorInput, remove the redundant numDimensions !== undefined check from the return expression because Number.isInteger(numDimensions) already handles undefined; update the return to rely on Number.isInteger(numDimensions) && numDimensions > 0 (kept) along with the existing isVectorComponent(values), isVectorComponent(indices) and values.length === indices.length checks so behavior remains the same.packages/core/test/unit/data-types/string-types.test.ts (1)
218-349: ⚡ Quick winAvoid gating dialect-agnostic VECTOR tests behind dialect support.
Wrapping the whole suite in the support check skips constructor and validation coverage on non-VECTOR dialect runs. Keep dialect-independent tests always on, and gate only assertions that truly require native VECTOR SQL support.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/core/test/unit/data-types/string-types.test.ts` around lines 218 - 349, The current test file gates the entire DataTypes.VECTOR suite behind sequelize.dialect.supports?.dataTypes?.VECTOR which hides constructor and validate tests on dialects without native VECTOR; move the dialect-agnostic describe blocks (the "constructor" and "validate" describes that call DataTypes.VECTOR, type.validate, and expect validations) outside of the if block so they always run, and keep only the dialect-specific "toSql" tests (the testDataTypeSql calls for DataTypes.VECTOR and DataTypes.VECTOR(4) etc.) inside the existing support check; update references to DataTypes.VECTOR and the describe titles accordingly so constructor/validate tests run unconditionally while SQL-generation tests remain gated.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@packages/core/test/unit/data-types/string-types.test.ts`:
- Around line 218-349: The current test file gates the entire DataTypes.VECTOR
suite behind sequelize.dialect.supports?.dataTypes?.VECTOR which hides
constructor and validate tests on dialects without native VECTOR; move the
dialect-agnostic describe blocks (the "constructor" and "validate" describes
that call DataTypes.VECTOR, type.validate, and expect validations) outside of
the if block so they always run, and keep only the dialect-specific "toSql"
tests (the testDataTypeSql calls for DataTypes.VECTOR and DataTypes.VECTOR(4)
etc.) inside the existing support check; update references to DataTypes.VECTOR
and the describe titles accordingly so constructor/validate tests run
unconditionally while SQL-generation tests remain gated.
In `@packages/oracle/src/_internal/data-types-overrides.ts`:
- Around line 549-565: In isOracleSparseVectorInput, remove the redundant
numDimensions !== undefined check from the return expression because
Number.isInteger(numDimensions) already handles undefined; update the return to
rely on Number.isInteger(numDimensions) && numDimensions > 0 (kept) along with
the existing isVectorComponent(values), isVectorComponent(indices) and
values.length === indices.length checks so behavior remains the same.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 6a316a46-a0cb-4e2a-96cf-30d68fa37caf
📒 Files selected for processing (19)
packages/core/src/abstract-dialect/data-types.tspackages/core/src/abstract-dialect/dialect.tspackages/core/src/abstract-dialect/query-interface.d.tspackages/core/src/data-types.tspackages/core/src/sequelize.d.tspackages/core/src/sequelize.jspackages/core/test/integration/dialects/oracle/vector.test.jspackages/core/test/unit/data-types/string-types.test.tspackages/core/test/unit/dialects/oracle/query-generator.test.jspackages/core/test/unit/dialects/oracle/vector.test.jspackages/core/test/unit/query-generator/show-indexes-query.test.tspackages/core/test/unit/sequelize.test.tspackages/oracle/src/_internal/data-types-overrides.tspackages/oracle/src/dialect.tspackages/oracle/src/query-generator-typescript.internal.tspackages/oracle/src/query-generator.internal.tspackages/oracle/src/query-generator.jspackages/oracle/src/query.jstest/esm-named-exports.test.js
993528c to
484a3d9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Checklist
Description of Changes
This PR adds
VECTORsupport with Oracle-first implementation, shared core abstractions, and full validation/test coverage for vector SQL generation paths.Core datatype & API groundwork
AbstractVECTORBase(shared behavior/validation)DataTypes.VECTORconstructor overloads:DataTypes.VECTORDataTypes.VECTOR(dimension)DataTypes.VECTOR(dimension, format)DataTypes.VECTOR({ dimension, format })cosineDistanceinnerProductl1Distancel2DistancevectorDistanceOracle integration
supports.dataTypes.VECTOR = true.SPARSE) and bind marshalling.usingdistanceaccuracyparametersorderTests
Guidance for other dialects
For future dialect integration, this PR establishes the recommended path:
supports.dataTypes.VECTOR.VECTORwhere generic option shape is sufficient, or extendAbstractVECTORBasefor dialect-specific option models.List of Breaking Changes
Summary by CodeRabbit
New Features
Database
Tests
Chores