[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visible to constrain output strides #155395

anijain2305 · Jun 7, 2025

Stack from ghstack (oldest at bottom):

-> [inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visible to constrain output strides #155395

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov

…ble to constrain output strides [ghstack-poisoned]

pytorch-bot · Jun 7, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/155395

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 11 Cancelled Jobs, 5 Pending, 2 Unrelated Failures

As of commit 4eb267e with merge base 1b03238 ():

NEW FAILURES - The following jobs have failed:

pull / linux-jammy-cuda12.6-py3.10-gcc11 / build (gh)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
pull / linux-jammy-cuda12.6-py3.10-gcc11-sm89 / build (gh)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

CANCELLED JOBS - The following jobs were cancelled. Please retry:

inductor / unit-test / cuda12.8-py3.12-gcc9-sm86 / build (gh)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
Lint / Link checks / lint-xrefs / linux-job (gh)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
Lint / Test tools / linux-job (gh)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
Lint / workflow-checks / linux-job (gh)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
pull / linux-jammy-cuda12.8-cudnn9-py3.9-clang12 / build (gh)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
pull / linux-jammy-py3.9-clang12 / build (gh)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
pull / linux-jammy-py3.9-clang12-onnx / build (gh)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
pull / linux-jammy-py3.9-gcc11-mobile-lightweight-dispatch-build / build (gh)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
pull / linux-jammy-py3.9-gcc11-no-ops / build (gh)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
pull / linux-jammy-py3.9-gcc11-pch / build (gh)
trunk / cuda12.8-py3.10-gcc9-sm80 / build (gh)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

pull / cuda12.8-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) (gh) (similar failure)
MISSING REGRESSION TEST
pull / linux-jammy-cuda12.6-py3.10-gcc11-test / test (distributed, 3, 3, linux.g4dn.12xlarge.nvidia.gpu) (gh) (matched linux rule in flaky-rules.json)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

…ble to constrain output strides ghstack-source-id: df83517 Pull Request resolved: #155395

…s user_visible to constrain output strides" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov [ghstack-poisoned]

…ble to constrain output strides ghstack-source-id: 99a532e Pull Request resolved: #155395

anijain2305 · Jun 7, 2025

cc @eellison @zou3519 What would be a good test for this? I tried a few options, but I am unable to write one out.

…s user_visible to constrain output strides" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov [ghstack-poisoned]

…ble to constrain output strides ghstack-source-id: 9a907cd Pull Request resolved: #155395

[ghstack-poisoned]

…ble to constrain output strides ghstack-source-id: 3c60cab Pull Request resolved: #155395

[ghstack-poisoned]

…ble to constrain output strides ghstack-source-id: 1378966 Pull Request resolved: #155395

[ghstack-poisoned]

…ble to constrain output strides ghstack-source-id: 3e0ad90 Pull Request resolved: #155395

[ghstack-poisoned]

…ble to constrain output strides ghstack-source-id: d0b67f9 Pull Request resolved: #155395

anijain2305 · Jun 9, 2025

Added the test, but I am unable to make it fail in the absence of this PR. In the absence of this PR, we just end up having a kernel that strides the output of the invoke_subgraph to the custom op.

[ghstack-poisoned]

…ble to constrain output strides ghstack-source-id: 2fc1b47 Pull Request resolved: #155395

zou3519

this looks reasonable but tbh I am not completely convinced it works since we can't come up with a test case that fails outside of this PR

anijain2305 · Jun 11, 2025

Yes, I am not convinced either. The logic makes sense. So, I will merge it.

I will try to work with some inductor folks to come up with some example.

anijain2305 · Jun 11, 2025

@pytorchbot merge

pytorchmergebot · Jun 11, 2025

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · Jun 11, 2025

Merge failed

Reason: Command git -C /home/runner/work/pytorch/pytorch cherry-pick -x 978f120389b12cbb43f9e92d71231e9aeb2cc99a returned non-zero exit code 1

Auto-merging test/higher_order_ops/test_invoke_subgraph.py
Auto-merging torch/_inductor/compile_fx.py
CONFLICT (content): Merge conflict in torch/_inductor/compile_fx.py
error: could not apply 978f120389b... [inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visible to constrain output strides
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".
hint: Disable this message with "git config set advice.mergeConflict false"

Details for Dev Infra team

Raised by workflow job

[ghstack-poisoned]

…ble to constrain output strides ghstack-source-id: fe61b2f Pull Request resolved: #155395

anijain2305 · Jun 11, 2025

@pytorchbot merge

pytorchmergebot · Jun 11, 2025

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · Jun 11, 2025

Merge failed

Reason: 11 mandatory check(s) failed. The first few are:

Dig deeper by viewing the failures on hud

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

anijain2305 · Jun 12, 2025

@pytorchbot merge -i

pytorchmergebot · Jun 12, 2025

Merge started

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

anijain2305 · Jun 12, 2025

@pytorchbot merge -f "broken ci"

pytorchmergebot · Jun 12, 2025

The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command
For more information see pytorch-bot wiki.

pytorchmergebot · Jun 12, 2025

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visi…

b7d4bbb

…ble to constrain output strides [ghstack-poisoned]

This was referenced Jun 7, 2025

[invoke_subgraph] Throw assertion on uncaptured speculate_subgraph #155270

Closed

[invoke_subgraph] Add logging #155284

Closed

[invoke_subgraph] Use eager input vals to constrain input strides #155291

Closed

pytorch-bot bot added ciflow/inductor module: inductor labels Jun 7, 2025

anijain2305 added a commit that referenced this pull request Jun 7, 2025

[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visi…

65232ec

…ble to constrain output strides ghstack-source-id: df83517 Pull Request resolved: #155395

anijain2305 added a commit that referenced this pull request Jun 7, 2025

[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visi…

abc2bdc

…ble to constrain output strides ghstack-source-id: 99a532e Pull Request resolved: #155395

anijain2305 added a commit that referenced this pull request Jun 8, 2025

[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visi…

b343b8a

…ble to constrain output strides ghstack-source-id: 9a907cd Pull Request resolved: #155395

anijain2305 added ciflow/trunk Trigger trunk jobs on your pull request topic: not user facing topic category labels Jun 9, 2025

Update

e2cfca8

[ghstack-poisoned]

anijain2305 added a commit that referenced this pull request Jun 9, 2025

[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visi…

4309125

…ble to constrain output strides ghstack-source-id: 3c60cab Pull Request resolved: #155395

Update

47d7e24

[ghstack-poisoned]

anijain2305 added a commit that referenced this pull request Jun 9, 2025

[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visi…

7c983c1

…ble to constrain output strides ghstack-source-id: 1378966 Pull Request resolved: #155395

Update

2697469

[ghstack-poisoned]

anijain2305 added a commit that referenced this pull request Jun 9, 2025

[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visi…

9e30f87

…ble to constrain output strides ghstack-source-id: 3e0ad90 Pull Request resolved: #155395

anijain2305 requested review from zou3519 and eellison June 9, 2025 21:18

anijain2305 added a commit that referenced this pull request Jun 9, 2025

[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visi…

ff70188

…ble to constrain output strides ghstack-source-id: 3e0ad90 Pull Request resolved: #155395

Update

ab44d5f

[ghstack-poisoned]

anijain2305 added a commit that referenced this pull request Jun 9, 2025

[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visi…

f218c8a

…ble to constrain output strides ghstack-source-id: d0b67f9 Pull Request resolved: #155395

Update

04b4286

[ghstack-poisoned]

anijain2305 added a commit that referenced this pull request Jun 10, 2025

[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visi…

978f120

…ble to constrain output strides ghstack-source-id: 2fc1b47 Pull Request resolved: #155395

zou3519 approved these changes Jun 11, 2025

View reviewed changes

pytorchmergebot added the merging label Jun 11, 2025

pytorchmergebot removed the merging label Jun 11, 2025

Update

4eb267e

[ghstack-poisoned]

anijain2305 added a commit that referenced this pull request Jun 11, 2025

[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visi…

c89f8ba

…ble to constrain output strides ghstack-source-id: fe61b2f Pull Request resolved: #155395

pytorchmergebot added the merging label Jun 11, 2025

pytorchmergebot removed the merging label Jun 11, 2025

pytorchmergebot added the merging label Jun 12, 2025

pytorchmergebot closed this in c9e9a0c Jun 12, 2025

pytorchmergebot added Merged and removed merging labels Jun 12, 2025

Search code, repositories, users, issues, pull requests...

[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visible to constrain output strides #155395

[inductor][invoke_subgraph] Mark invoke_subgraph outputs as user_visible to constrain output strides #155395

Uh oh!

Conversation

anijain2305 commented Jun 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Jun 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/155395

❌ 2 New Failures, 11 Cancelled Jobs, 5 Pending, 2 Unrelated Failures

Uh oh!

anijain2305 commented Jun 7, 2025

Uh oh!

anijain2305 commented Jun 9, 2025

Uh oh!

zou3519 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anijain2305 commented Jun 11, 2025

Uh oh!

anijain2305 commented Jun 11, 2025

Uh oh!

pytorchmergebot commented Jun 11, 2025

Merge started

Uh oh!

pytorchmergebot commented Jun 11, 2025

Merge failed

Uh oh!

anijain2305 commented Jun 11, 2025

Uh oh!

pytorchmergebot commented Jun 11, 2025

Merge started

Uh oh!

pytorchmergebot commented Jun 11, 2025

Merge failed

Uh oh!

anijain2305 commented Jun 12, 2025

Uh oh!

pytorchmergebot commented Jun 12, 2025

Merge started

Uh oh!

anijain2305 commented Jun 12, 2025

Uh oh!

pytorchmergebot commented Jun 12, 2025

Uh oh!

pytorchmergebot commented Jun 12, 2025

Merge started

Uh oh!

Uh oh!

anijain2305 commented Jun 7, 2025 •

edited

Loading

pytorch-bot bot commented Jun 7, 2025 •

edited

Loading

zou3519 left a comment •

edited

Loading