feat(kernel): consume the folded directResults execute()#837
Closed
msrathore-db wants to merge 1 commit into
maindatabricks/databricks-sql-python:mainfrom
msrathore/use-kernel-directresultsdatabricks/databricks-sql-python:msrathore/use-kernel-directresultsCopy head branch name to clipboard
Closed
feat(kernel): consume the folded directResults execute()#837msrathore-db wants to merge 1 commit intomaindatabricks/databricks-sql-python:mainfrom msrathore/use-kernel-directresultsdatabricks/databricks-sql-python:msrathore/use-kernel-directresultsCopy head branch name to clipboard
msrathore-db wants to merge 1 commit into
maindatabricks/databricks-sql-python:mainfrom
msrathore/use-kernel-directresultsdatabricks/databricks-sql-python:msrathore/use-kernel-directresultsCopy head branch name to clipboard
Conversation
The kernel's `Statement.execute()` is now the directResults call (databricks/databricks-sql-kernel#136): it returns a SINGLE `ExecutedAsyncStatement` -- seeded with the inline result when the query finished within the server inline wait (fast path, zero extra round-trips), or a poll/cancel handle when still running -- instead of always blocking to terminal. `KernelDatabricksClient.execute_command` now drives ONE uniform path (no `hasattr(await_result)` arm to feature-detect): register the handle (so `cursor.cancel()` / `close()` reach it), then `await_result()` to drive it to a ready result set -- preserving execute()'s blocking contract. `cursor.execute()` behaviour is unchanged for callers (still blocks to a ready result set); this just adapts to the kernel's new single-handle return. Mid-run cancel still works via the `_sync_cancellers` canceller registered before execute. Tested e2e against live warehouses (auto-closing and non-auto-closing): SELECT 1, range(N), and CREATE+count all succeed on both. The kernel maps an auto-closed `CLOSED` statement to `Succeeded`, so the connector sees a uniform success regardless of warehouse auto-close behaviour. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
acecd76 to
8974e7b
Compare
Contributor
Author
|
Not needed with the additive kernel approach (databricks/databricks-sql-kernel#140). That PR adds execute_direct() alongside an unchanged blocking execute(), so use_kernel keeps using execute() exactly as today — no client.py change. Verified e2e against the additive kernel on the unchanged upstream/main driver: CREATE commits, close() cheap (no close-drive), mid-run cancel works (258ms, via the existing _sync_cancellers canceller). The await_result adaptation here was only required for the in-place fold (kernel #136, closed). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adapt
KernelDatabricksClient.execute_commandto the kernel's folded directResultsexecute()(databricks/databricks-sql-kernel#136), which now returns a singleExecutedAsyncStatement— seeded with the inline result when the query finished within the server inline wait (fast path, zero extra round-trips), or a poll/cancel handle when still running — instead of always blocking to terminal.The sync path now drives one uniform flow (no
hasattr(await_result)arm to feature-detect): register the handle (socursor.cancel()/close()reach it), thenawait_result()to drive it to a ready result set — preservingexecute()'s blocking contract.Caller-visible behaviour
cursor.execute()is unchanged for callers (still blocks to a ready result set); this just adapts to the kernel's new single-handle return. Mid-run cancel still works via the_sync_cancellerscanceller registered before execute.Testing
Tested e2e against live warehouses (auto-closing and non-auto-closing):
SELECT 1,range(N), andCREATE+count all succeed on both. The kernel maps an auto-closedCLOSEDstatement toSucceeded, so the connector sees a uniform success regardless of warehouse auto-close behaviour.This pull request and its description were written by Isaac.