asyncio split part 2#32
asyncio split part 2#32filintod wants to merge 11 commits intodapr:maindapr/durabletask-python:mainfrom filintod:filinto/asyncio-p2filintod/durabletask-python:filinto/asyncio-p2Copy head branch name to clipboard
Conversation
b20de61 to
27068dc
Compare
Signed-off-by: Filinto Duran <1373693+filintod@users.noreply.github.com>
27068dc to
0126d47
Compare
Signed-off-by: Filinto Duran <1373693+filintod@users.noreply.github.com>
Signed-off-by: Filinto Duran <1373693+filintod@users.noreply.github.com>
| - `DAPR_GRPC_ENDPOINT` - Full endpoint (e.g., `localhost:4001`, `grpcs://host:443`) | ||
| - `DAPR_GRPC_HOST` (or `DAPR_RUNTIME_HOST`) and `DAPR_GRPC_PORT` - Host and port separately | ||
|
|
||
| Example (common ports: 4001 for DurableTask-Go emulator, 50001 for Dapr sidecar): |
There was a problem hiding this comment.
is the 4001 supposed to be for this repo, durabletask python? or is there some emulator thing somewhere?
There was a problem hiding this comment.
I guess this is historical, but 4001 was the default before. also look at durabletask-go https://github.com/dapr/durabletask-go (same 4001)
README.md
Outdated
|
|
||
| Optional sandbox mode (`best_effort` or `strict`) patches `asyncio.sleep`, `random`, `uuid.uuid4`, and `time.time` within the workflow step to deterministic equivalents. This is best-effort and not a correctness guarantee. | ||
|
|
||
| In `strict` mode, `asyncio.create_task` is blocked inside workflows to preserve determinism and will raise a `SandboxViolationError` if used. |
There was a problem hiding this comment.
i see you say to presever determinism, but i still dont understand the why here. can you expand the explanation pls
| class NonDeterminismWarning(UserWarning): | ||
| """Warning raised when non-deterministic functions are detected in workflows.""" | ||
|
|
||
| pass |
There was a problem hiding this comment.
is this used if it has pass in it? or its used and used through the inheritance?
| python your_workflow.py | ||
|
|
||
| # Production mode (no warnings, optimal performance) | ||
| unset DAPR_WF_DEBUG | ||
| python your_workflow.py |
There was a problem hiding this comment.
dont these need to be wrapped in a dapr run cmd then if they req a sidecar?
| @@ -0,0 +1,301 @@ | ||
| # Durable Task AsyncIO Internals | ||
|
|
||
| This document explains how the AsyncIO implementation in this repository integrates with the existing generator‑based Durable Task runtime. It covers the coroutine→generator bridge, awaitable design, sandboxing and non‑determinism detection, error/cancellation semantics, debugging, and guidance for extending the system. |
There was a problem hiding this comment.
is the coroutine-> generator bridge the sandbox thing? or different? what does non-determinsitm detection mean? and if triggered then do we err?
| Key modules: | ||
| - `durabletask/aio/context.py` — Async workflow context and deterministic utilities | ||
| - `durabletask/aio/driver.py` — Coroutine→generator bridge | ||
| - `durabletask/aio/sandbox.py` — Scoped patching and non‑determinism detection |
There was a problem hiding this comment.
is it worth separating the non-determinism detection stuff then into a diff file? or put the non-determinism stuff with the deterministic utilities from context.py?
|
|
||
| ### Coroutine→Generator Bridge | ||
|
|
||
| Async orchestrators are authored as `async def` but executed by Durable Task as generators that yield `durabletask.task.Task` (or composite) instances. The bridge implements a driver that manually steps a coroutine and converts each `await` into a yielded Durable Task operation. |
There was a problem hiding this comment.
can you pls add in how this is the translation of async world to safe sync workflow generator world pls
There was a problem hiding this comment.
and is it the case that each activity yields its own coroutine and then the workflow is in a main coroutine - just to make sure im tracking 🙏
| Concurrency: | ||
| - `when_all([...])` returns an awaitable that completes with a list of results | ||
| - `when_any([...])` returns an awaitable that completes with the first completed child | ||
| - `when_any_with_result([...])` returns `(index, result)` |
There was a problem hiding this comment.
are these just docs on existing methods? or did you add things for when_all? it not then can we pls rm?
| - Operation history when debug is enabled (`DAPR_WF_DEBUG=true` or `DT_DEBUG=true`) | ||
| - `get_debug_info()` to inspect state for diagnostics |
There was a problem hiding this comment.
probably can rm these pls bc i think setting log level is sufficient already right?
There was a problem hiding this comment.
maybe the name is not correct, this is more like getting a snapshot of the context info but mostly for debugging/exception
Signed-off-by: Filinto Duran <1373693+filintod@users.noreply.github.com>
…RMINISTIC_DETECTION Signed-off-by: Filinto Duran <1373693+filintod@users.noreply.github.com>
Co-authored-by: Sam <sam@diagrid.io> Signed-off-by: Filinto Duran <1373693+filintod@users.noreply.github.com>
Signed-off-by: Filinto Duran <1373693+filintod@users.noreply.github.com>
|
@filintod this PR is just too massive to even get a grasp on. I suggest it to be split into substantially smaller bite sizes and then implemented in iterations. @sicoyle, @acroca if you agree then I think we should close this one and let it arrive in smaller chunks |
|
I agree. |
No description provided.