[dynamo] Mark a vt unspecialized nn module variable source earlier #154780

anijain2305 · May 31, 2025

Stack from ghstack (oldest at bottom):

I am working on providing some skip guard helper functions to allow users to reduce guard overhead. This is a refactor to allow that.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @amjames

[ghstack-poisoned]

pytorch-bot · May 31, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/154780

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 3285033 with merge base af9f18e ():

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

pull / linux-jammy-py3-clang12-executorch / build (gh) (#150261)
Final attempt failed. Child_process exited with error code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

… earlier" I am working on providing some skip guard helper functions to allow users to reduce guard overhead. This is a refactor to allow that. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

anijain2305 · Jun 1, 2025

torch/_dynamo/variables/builder.py

-                result = UnspecializedBuiltinNNModuleVariable(value, source=self.source)
+            if (
+                value.__module__.startswith(("torch.nn.modules", "torch.ao."))
+                and not value.__module__.startswith("torch.nn.modules.container")


container is deliberately left out to be bulitin because a container, like Sequential, can consist of user defined module.

… earlier" I am working on providing some skip guard helper functions to allow users to reduce guard overhead. This is a refactor to allow that. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames [ghstack-poisoned]

ghstack-source-id: f229e45 Pull Request resolved: #154780 [dynamo] Wrap unspec module with a unspec source earlier ghstack-source-id: f229e45 Pull Request resolved: #154789

StrongerXi · Jun 2, 2025

benchmarks/dynamo/ci_expected_accuracy/cpu_inductor_torchbench_inference.csv

-hf_BigBird,pass,18
+hf_BigBird,pass,24


Is this an increase in # of graph breaks?

Its actually a recompilation - I added #154867 so that users would know how to avoid the recompilation.

It happens for something like this

class Mod(torch.nn.Module): def __init__(self): super().__init__() self.c = 0 def forward(self, x): self.c += 1 return x * self.c

You can say this is somehow BC breaking. But there was a inconsistency earlier where sometimes we would consider nn.Module int atttibutes static and sometimes symint. This PR makes it more precise, where we always consider it static.

Why not always make it symint? I tried this and just a simple symint change here brings lots of dynamism into the model, causing crashes in a couple of examples I saw. In most cases, when users think of dynamism, they want their input tensors to be dynamic. So a general heuristic that nn module integer attributes are static by default makes more sense, with an option to override it - allow_unspec_int_on_nn_module. From code comments, this seems like the original intent as well, but there were a few bugs/inconsistencies.

cc @bobrenjc93 @zou3519

torch/_dynamo/variables/builder.py

pytorchmergebot · Jun 3, 2025

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

…ytorch#154780) I am working on providing some skip guard helper functions to allow users to reduce guard overhead. This is a refactor to allow that. Pull Request resolved: pytorch#154780 Approved by: https://github.com/StrongerXi, https://github.com/jansel

…#154867) For program like this ``` class Mod(torch.nn.Module): def __init__(self): super().__init__() self.c = 0 def forward(self, x): self.c += 1 return x * self.c ``` You can check the recompile reasons at https://manifold.edge.x2p.facebook.net/v0/read/tree/logs/.tmpzv9z6Q/index.html?bucketName=tlparse_reports&apiKey=tlparse_reports-key&withPayload=1&timeoutMsec=10000 ![image](https://github.com/user-attachments/assets/856a95fd-0533-4abc-a213-1f73ae2cb766) Pull Request resolved: #154867 Approved by: https://github.com/zou3519 ghstack dependencies: #154780

…rlier Reland of #154780

…ytorch#154780) I am working on providing some skip guard helper functions to allow users to reduce guard overhead. This is a refactor to allow that. Pull Request resolved: pytorch#154780 Approved by: https://github.com/StrongerXi, https://github.com/jansel

…pytorch#154867) For program like this ``` class Mod(torch.nn.Module): def __init__(self): super().__init__() self.c = 0 def forward(self, x): self.c += 1 return x * self.c ``` You can check the recompile reasons at https://manifold.edge.x2p.facebook.net/v0/read/tree/logs/.tmpzv9z6Q/index.html?bucketName=tlparse_reports&apiKey=tlparse_reports-key&withPayload=1&timeoutMsec=10000 ![image](https://github.com/user-attachments/assets/856a95fd-0533-4abc-a213-1f73ae2cb766) Pull Request resolved: pytorch#154867 Approved by: https://github.com/zou3519 ghstack dependencies: pytorch#154780

seemethere · Jun 4, 2025

@pytorchbot revert -c ghfirst -m "This fails internal testing see, https://fburl.com/diff/b0yuxk4w"

pytorchmergebot · Jun 4, 2025

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

…tributes (#154867)" This reverts commit 6c2f941. Reverted #154867 on behalf of https://github.com/seemethere due to This fails internal testing see, https://fburl.com/diff/b0yuxk4w ([comment](#154780 (comment)))

…rlier (#154780)" This reverts commit cc96feb. Reverted #154780 on behalf of https://github.com/seemethere due to This fails internal testing see, https://fburl.com/diff/b0yuxk4w ([comment](#154780 (comment)))

pytorchmergebot · Jun 4, 2025

@anijain2305 your PR has been successfully reverted.

…rlier Reland of #154780

anijain2305 · Jun 4, 2025

Relanding at #155099

Summary: Reland of pytorch/pytorch#154780 Fixes #ISSUE_NUMBER cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang amjames X-link: pytorch/pytorch#155099 Reviewed By: williamwen42 Differential Revision: D75928791 Pulled By: anijain2305 fbshipit-source-id: 55f8a214e3b0ff573dd1bdd1fb5fb34a220a620d

…rlier (#155099) Reland of #154780 Fixes #ISSUE_NUMBER Pull Request resolved: #155099 Approved by: https://github.com/williamwen42

…ytorch#154780) I am working on providing some skip guard helper functions to allow users to reduce guard overhead. This is a refactor to allow that. Pull Request resolved: pytorch#154780 Approved by: https://github.com/StrongerXi, https://github.com/jansel

…pytorch#154867) For program like this ``` class Mod(torch.nn.Module): def __init__(self): super().__init__() self.c = 0 def forward(self, x): self.c += 1 return x * self.c ``` You can check the recompile reasons at https://manifold.edge.x2p.facebook.net/v0/read/tree/logs/.tmpzv9z6Q/index.html?bucketName=tlparse_reports&apiKey=tlparse_reports-key&withPayload=1&timeoutMsec=10000 ![image](https://github.com/user-attachments/assets/856a95fd-0533-4abc-a213-1f73ae2cb766) Pull Request resolved: pytorch#154867 Approved by: https://github.com/zou3519 ghstack dependencies: pytorch#154780

…tributes (pytorch#154867)" This reverts commit 6c2f941. Reverted pytorch#154867 on behalf of https://github.com/seemethere due to This fails internal testing see, https://fburl.com/diff/b0yuxk4w ([comment](pytorch#154780 (comment)))

…rlier (pytorch#154780)" This reverts commit cc96feb. Reverted pytorch#154780 on behalf of https://github.com/seemethere due to This fails internal testing see, https://fburl.com/diff/b0yuxk4w ([comment](pytorch#154780 (comment)))

…rlier (pytorch#155099) Reland of pytorch#154780 Fixes #ISSUE_NUMBER Pull Request resolved: pytorch#155099 Approved by: https://github.com/williamwen42

[dynamo] Mark a vt unspecialized nn module variable source earlier

ee2fe18

[ghstack-poisoned]

anijain2305 mentioned this pull request May 31, 2025

[dynamo][guards] Prevent LENGTH guard on nn modules #154763

Closed

pytorch-bot bot added ciflow/inductor module: dynamo labels May 31, 2025

This was referenced May 31, 2025

[dynamo] Record the pre-graph bytecode using fast record function event #154769

Closed

[dynamo][guards] Flush cache to more accurately measure guard overhead #154764

Closed

anijain2305 added ciflow/trunk Trigger trunk jobs on your pull request topic: not user facing topic category labels May 31, 2025

anijain2305 mentioned this pull request May 31, 2025

[dynamo] Wrap unspec module with a unspec source earlier #154789

Closed

anijain2305 added 3 commits May 30, 2025 23:18

anijain2305 commented Jun 1, 2025

View reviewed changes

anijain2305 added 3 commits June 1, 2025 00:51

anijain2305 requested a review from zou3519 as a code owner June 1, 2025 17:10

anijain2305 added 4 commits June 1, 2025 14:50

anijain2305 added the ci-no-td Do not run TD on this PR label Jun 2, 2025

anijain2305 requested review from jansel, williamwen42 and StrongerXi June 2, 2025 14:38

StrongerXi approved these changes Jun 2, 2025

View reviewed changes

pytorchmergebot added the Merged label Jun 3, 2025

pytorchmergebot closed this in cc96feb Jun 3, 2025

pytorchmergebot removed the merging label Jun 3, 2025

anijain2305 mentioned this pull request Jun 4, 2025

[dynamo] Provide helper functions for guard filter hook #155083

Open

anijain2305 added a commit that referenced this pull request Jun 4, 2025

[reland][dynamo] Mark a vt unspecialized nn module variable source ea…

1b7f9f2

…rlier Reland of #154780

anijain2305 mentioned this pull request Jun 4, 2025

[reland][dynamo] Mark a vt unspecialized nn module variable source earlier #155099

Closed

anijain2305 added a commit that referenced this pull request Jun 4, 2025

[reland][dynamo] Mark a vt unspecialized nn module variable source ea…

e4a2ed0

…rlier Reland of #154780

pytorchmergebot added the Reverted label Jun 4, 2025

pytorchmergebot reopened this Jun 4, 2025

anijain2305 added a commit that referenced this pull request Jun 4, 2025

[reland][dynamo] Mark a vt unspecialized nn module variable source ea…

dfeb6ba

…rlier Reland of #154780

anijain2305 closed this Jun 4, 2025

pytorchmergebot pushed a commit that referenced this pull request Jun 4, 2025

[reland][dynamo] Mark a vt unspecialized nn module variable source ea…

c881f2d

…rlier (#155099) Reland of #154780 Fixes #ISSUE_NUMBER Pull Request resolved: #155099 Approved by: https://github.com/williamwen42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[dynamo] Mark a vt unspecialized nn module variable source earlier #154780

[dynamo] Mark a vt unspecialized nn module variable source earlier #154780

Uh oh!

anijain2305 commented May 31, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented May 31, 2025 •

edited

Loading

Uh oh!

anijain2305 Jun 1, 2025

Uh oh!

StrongerXi Jun 2, 2025

Uh oh!

anijain2305 Jun 2, 2025

Uh oh!

Uh oh!

Uh oh!

pytorchmergebot commented Jun 3, 2025

Uh oh!

seemethere commented Jun 4, 2025

Uh oh!

pytorchmergebot commented Jun 4, 2025

Uh oh!

pytorchmergebot commented Jun 4, 2025

Uh oh!

anijain2305 commented Jun 4, 2025

Uh oh!

Uh oh!

		hf_BigBird,pass,18
		hf_BigBird,pass,24

Search code, repositories, users, issues, pull requests...

[dynamo] Mark a vt unspecialized nn module variable source earlier #154780

[dynamo] Mark a vt unspecialized nn module variable source earlier #154780

Uh oh!

Conversation

anijain2305 commented May 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented May 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/154780

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

anijain2305 Jun 1, 2025

Choose a reason for hiding this comment

Uh oh!

StrongerXi Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

anijain2305 Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pytorchmergebot commented Jun 3, 2025

Merge started

Uh oh!

seemethere commented Jun 4, 2025

Uh oh!

pytorchmergebot commented Jun 4, 2025

Uh oh!

pytorchmergebot commented Jun 4, 2025

Uh oh!

anijain2305 commented Jun 4, 2025

Uh oh!

Uh oh!

anijain2305 commented May 31, 2025 •

edited

Loading

pytorch-bot bot commented May 31, 2025 •

edited

Loading