SEA: Normalise Columns returned by Metadata Queries#627
Closed
varun-edachali-dbx wants to merge 316 commits into
Closed
SEA: Normalise Columns returned by Metadata Queries#627varun-edachali-dbx wants to merge 316 commits into
varun-edachali-dbx wants to merge 316 commits into
Commits
This pull request is big! We're only showing the most recent 250 commits
Commits on Jun 11, 2025
remove filtering, metadata ops
Show description for 484064ecommittedraise NotImplementedErrror for metadata ops
Show description for 030edf8committedalign SeaResultSet with new structure
Show description for 4e07f1ecommittedcorrect sea res set tests
Show description for 65e7c6bcommittedadd metadata commands
Show description for 30f8266committedformatting (black)
Show description for 033ae73committedadd metadata command unit tests
Show description for 33821f4committedminimal fetch phase intro
Show description for 71b451acommitted- committed
- committed
working JSON + INLINE
Show description for c038d5acommittedchange to valid table name
Show description for 3e22c6ccommittedrmeove redundant queue init
Show description for 716304bcommittedlarge query results
Show description for e96e5b8committed- committed
remove un-necessary changes
Show description for 165c4f3committedsimplify test module
Show description for a6e40d0committedlogging -> debug level
Show description for 52e3088committedchange table name in log
Show description for 641c09bcommitted- committed
remove un-necessary changes
Show description for ffded6ecommittedremove un-necessary backend cahnges
Show description for 227f6b3committedremove un-needed GetChunksResponse
Show description for 68657a3committedremove un-needed GetChunksResponse
Show description for 3940eeccommittedreduce code duplication in response parsing
Show description for 37813bacommittedreduce code duplication
Show description for 267c9f4committedmore clear docstrings
Show description for 2967119committedintroduce strongly typed ChunkInfo
Show description for 47fd60dcommitted
Commits on Jun 12, 2025
remove is_volume_operation from response
Show description for 982fdf2committedadd is_volume_op and more ResultData fields
Show description for 9e14d48committed- committed
- committed
add test scripts
Show description for 05ee4e7committed- committed
Revert "Merge branch 'sea-migration' into exec-models-sea"
Show description for 2952d8dcommitted- committed
Revert "Merge branch 'exec-models-sea' into exec-phase-sea"
Show description for cbace3fcommittedchange logging level
Show description for c075b07committedremove un-necessary changes
Show description for c62f76dcommittedremove excess changes
Show description for 199402ecommittedremove excess changes
Show description for 8ac574bcommittedMerge branch 'sea-migration' into exec-phase-sea
Show description for 398ca70committedremove _get_schema_bytes (for now)
Show description for b1acc5bcommittedredundant comments
Show description for ef2a7eecommitted- committed
remove fetch phase methods
Show description for af8f74ecommittedreduce code repetititon + introduce gaps after multi line pydocs
Show description for 5540c5ccommittedremove unused imports
Show description for efe3881committedmove description extraction to helper func
Show description for 36ab59bcommittedformatting (black)
Show description for 1d57c99committedadd more unit tests
Show description for df6dac2committedstreamline unit tests
Show description for ad0e527committedtest getting the list of allowed configurations
Show description for ed446a0committedreduce diff
Show description for 38e4b5ccommittedreduce diff
Show description for 94879c0committed
Commits on Jun 13, 2025
house constants in enums for readability and immutability
Show description for 1809956committedadd note on hybrid disposition
Show description for da5260ccommitted
Commits on Jun 16, 2025
[squashed from cloudfetch-sea] introduce external links + arrow functionality
Show description for 6ec265fcommittedreduce responsibility of Queue
Show description for b2ad5e6committedreduce repetition in arrow tablee creation
Show description for 66d0df6committedreduce redundant code in CloudFetchQueue
Show description for eb7ec80committedmove chunk link progression to separate func
Show description for a3a8a4acommittedremove redundant log
Show description for ea79bc8committedimprove logging
Show description for 5b49405committedremove reliance on schema_bytes in SEA
Show description for 015fb76committedremove redundant note on arrow_schema_bytes
Show description for 0385ffbcommitteduse more fetch methods
Show description for 5380c7acommittedremove redundant schema_bytes from parent constructor
Show description for 27b781fcommittedonly call get_chunk_link with non null chunk index
Show description for 238dc0acommittedalign SeaResultSet structure with ThriftResultSet
Show description for b3bb07ecommittedremvoe _fill_result_buffer from SeaResultSet
Show description for 13e6346committedreduce code repetition
Show description for f90b4d4committedalign SeaResultSet with ext-links-sea
Show description for 23963fccommittedremove redundant methods
Show description for dd43715committedupdate unit tests
Show description for 34a7f66committedremove accidental venv changes
Show description for 715cc13committed
Commits on Jun 17, 2025
pre-fetch next chunk link on processing current
Show description for fb53dd9committedreduce nesting
Show description for d893877committedline break after multi line pydoc
Show description for a165f1ccommittedre-introduce schema_bytes for better abstraction (likely temporary)
Show description for d68e4eacommitted- committed
add fetchmany_arrow and fetchall_arrow
Show description for a0705bccommitted- committed
remove accidental changes in sea backend tests
Show description for f7c11b9committed- committed
remove irrelevant changes
Show description for 6229848committedremove un-necessary test changes
Show description for fd52356committedremove un-necessary changes in thrift backend tests
Show description for 64e58b0committedremove unimplemented methods test
Show description for 2903473committed- committed
remove unimplemented method tests
Show description for 021ff4ccommittedmodify example scripts to include fetch calls
Show description for adecd53committed- committed
add GetChunksResponse
Show description for e3cef5ccommittedremove changes to sea test
Show description for ac50669committedre-introduce accidentally removed description extraction method
Show description for 03cdc4fcommittedfix type errors (ssl_options, CHUNK_PATH_WITH_ID..., etc.)
Show description for e1842d8committedaccess ssl_options through connection
Show description for 89a46afcommittedDEBUG level
Show description for 1d0b28bcommittedremove explicit multi chunk test
Show description for c8820d4committedmove cloud fetch queues back into utils.py
Show description for fe47787committedremove excess docstrings
Show description for 74f59b7committedmove ThriftCloudFetchQueue above SeaCloudFetchQueue
Show description for 4b456b2committedfix sea connector tests
Show description for bfc1f01committed- committed
correct patch module path in cloud fetch queue tests
Show description for 4883affcommittedremove unimplemented methods test
Show description for 0a2cdfdcommittedcorrect add_link docstring
Show description for cd3378ccommitted- committed
remove invalid import
Show description for cd22389committed- committed
- committed
- committed
- committed
Commits on Jun 18, 2025
better align queries with JDBC impl
Show description for 5ab9bbecommittedline breaks after multi-line PRs
Show description for 1ab6e87committedremove unused imports
Show description for f469c24committedfix: introduce ExecuteResponse import
Show description for 68ec65fcommitted- committed
remove unimplemented metadata methods test, un-necessary imports
Show description for f6d873dcommittedintroduce unit tests for metadata methods
Show description for 28675f5committed
Commits on Jun 20, 2025
remove verbosity in ResultSetFilter docstring
Show description for 3578659andauthoredremove un-necessary info in ResultSetFilter docstring
Show description for 8713023committedremove explicit type checking, string literals around forward annotations
Show description for 22dc252committedhouse SQL commands in constants
Show description for 390f592committed
Commits on Jun 23, 2025
convert complex types to string if not _use_arrow_native_complex_types
Show description for dd7dc6acommitted- committed
introduce unit tests for altered functionality
Show description for 2712d1ccommitted- committed
Revert "Merge branch 'fetch-json-inline' into ext-links-sea"
Show description for 48ad7b3committedreduce verbosity of ResultSetFilter docstring
Show description for a1f9b9ccommittedremove unused imports
Show description for 984e8eecommitted- committed
Revert "Merge branch 'fetch-json-inline' into ext-links-sea"
Show description for c313c2bcommittedRevert "reduce verbosity of ResultSetFilter docstring"
Show description for 3bc615ecommittedReapply "Merge branch 'fetch-json-inline' into ext-links-sea"
Show description for b6e1a10committedRevert "Merge branch 'fetch-json-inline' into ext-links-sea"
Show description for 2df3d39committedremove un-necessary filters changes
Show description for 5e75fb5committedremove un-necessary backend changes
Show description for 20822e4committedremove constants changes
Show description for 802d045committedremove changes in filters tests
Show description for f3f795acommittedremove unit test backend and JSON queue changes
Show description for f6c5950committedremove changes in sea result set testing
Show description for d210ccdcommittedRevert "remove changes in sea result set testing"
Show description for 22a953ecommittedRevert "remove unit test backend and JSON queue changes"
Show description for 3aed144committedRevert "remove changes in filters tests"
Show description for 0fe4da4committedRevert "remove constants changes"
Show description for 0e3c0a1committedRevert "remove un-necessary backend changes"
Show description for 93edb93committedRevert "remove un-necessary filters changes"
Show description for 871a44fcommittedremove unused imports
Show description for 0ce144dcommitted- committed
working version
Show description for 8c5cc77committedadopy _wait_until_command_done
Show description for 7f5c715committedintroduce metadata commands
Show description for 9ef5fadcommitteduse new backend structure
Show description for 44183dbcommittedconstrain backend diff
Show description for d59b351committedremove changes to filters
Show description for 1edc80acommittedmake _parse methods in models internal
Show description for f82658acommittedreduce changes in unit tests
Show description for 54eb0a4committed
Commits on Jun 24, 2025
run small queries with SEA during integration tests
Show description for 50cc1e2committedrun some tests for sea
Show description for 242307acommitted
Commits on Jun 25, 2025
allow empty schema bytes for alignment with SEA
Show description for 8a138e8committedpass is_vl_op to Sea backend ExecuteResponse
Show description for 82f9d6bcommitted
Commits on Jun 26, 2025
remove catalog requirement in get_tables
Show description for 35f1ef0committedmove filters.py to SEA utils
Show description for a515d26committedensure SeaResultSet
Show description for 59b1330committed- committed
prevent circular imports
Show description for dd40bebcommittedremove unused imports
Show description for 14057accommittedremove cast, throw error if not SeaResultSet
Show description for a4d5bdbcommitted- committed
pass param as TSparkParameterValue
Show description for eb1a9b4committed- committed
remove failing test (temp)
Show description for a3ca7c7committedremove SeaResultSet type assertion
Show description for 2c22010committedchange errors to align with spec, instead of arbitrary ValueError
Show description for c09508ecommittedmake SEA backend methods return SeaResultSet
Show description for e9b1314committeduse spec-aligned Exceptions in SEA backend
Show description for 8ede414committedremove defensive row type check
Show description for 09a1b11committed- committed
- committed
raise ProgrammingError for invalid id
Show description for a026d31committedmake is_volume_operation strict bool
Show description for 4446a9ecommittedremove complex types code
Show description for 138359dcommittedRevert "remove complex types code"
Show description for b99d0c4committed
Commits on Jun 27, 2025
introduce type conversion for primitive types for JSON + INLINE
Show description for 21c389dcommitted- committed
remove SEA running on metadata queries (known failures
Show description for 9f0f969committedremove un-necessary docstrings
Show description for 04a1936committedalign expected types with databricks sdk
Show description for 278b8cdcommittedlink rest api reference to validate types
Show description for 91b7f7fcommittedremove test_catalogs_returns_arrow_table test
Show description for 7a5ae13committedfix fetchall_arrow and fetchmany_arrow
Show description for f1776f3committedremove thrift aligned test_cancel_during_execute from SEA tests
Show description for 6143331committed- committed
remove un-necessary changes in example scripts
Show description for 5eaded4committedremove un-necessary chagnes in example scripts
Show description for eeed9a1committed_convert_json_table -> _create_json_table
Show description for f233886committedremove accidentally removed test
Show description for 68ac437committedremove new unit tests (to be re-added based on new arch)
Show description for 7fd0845committedremove changes in sea_result_set functionality (to be re-added)
Show description for ea7ff73committedintroduce more integration tests
Show description for 563da71committedremove SEA tests in parameterized queries
Show description for a018273committedremove partial parameter fix changes
Show description for c0e98f4committedremove un-necessary timestamp tests
Show description for 7343035committedslightly stronger typing of _convert_json_types
Show description for ec500b6committedstronger typing of json utility func s
Show description for 0b3e91dcommittedstronger typing of fetch*_json
Show description for 7664e44committedremove unused helper methods in SqlType
Show description for db7b8e5committedline breaks after multi line pydocs, remove excess logs
Show description for f75f2b5committedline breaks after multi line pydocs, reduce diff of redundant changes
Show description for e2d4ef5committedreduce diff of redundant changes
Show description for 21e3078committedmandate ResultData in SeaResultSet constructor
Show description for bb015e6committed- committed
remove complex type conversion
Show description for b3273c7committedcorrect fetch*_arrow
Show description for 38c2b88committed
Commits on Jul 3, 2025
- committed
recover old sea tests
Show description for fa2359dcommittedmove queue and result set into SEA specific dir
Show description for c07f709committedpass ssl_options into CloudFetchQueue
Show description for 9e4ef2ecommittedreduce diff
Show description for b00c06ccommittedremove redundant conversion.py
Show description for 10f55f0committedfix type issues
Show description for cd119e9committedValueError not ProgrammingError
Show description for d79638bcommittedreduce diff
Show description for f84578acommittedintroduce SEA cloudfetch e2e tests
Show description for c621c0ccommittedallow empty cloudfetch result
Show description for 7958cd9committedadd unit tests for CloudFetchQueue and SeaResultSet
Show description for e2d17ffcommittedskip pyarrow dependent tests
Show description for d348b35committed- committed
Commits on Jul 4, 2025
simplify download process: no pre-fetching
Show description for 4bd290ecommittedcorrect class name in logs
Show description for dfbbf79committed
Commits on Jul 7, 2025
introduce column normalisation for SEA metadat queries
Show description for 548e395committedadd refs, correct COLUMN_COLUMNS mappings
Show description for 61a9200committedremove TYPE_NAME mapping
Show description for c5d97bdcommittedmove test-column-mapping into tests/unit/
Show description for 48bb680committedformatting (black)
Show description for 8f12c33committed- committed
align with old impl
Show description for a5e9cdfcommittedalign next_n_rows with prev imple
Show description for be16634committedalign remaining_rows with prev impl
Show description for 6ec8656committedremove un-necessary Optional params
Show description for 7ea7b75committedremove un-necessary changes in thrift field if tests
Show description for 64be07bcommittedremove unused imports
Show description for 165644ccommitted
Commits on Jul 8, 2025
run large queries
Show description for abef941committed- committed
Commits on Jul 19, 2025
- committed
remove excess changes
Show description for f76d740committedtest_get_catalogs
Show description for 004bccecommitted