[nativert] Move Pytree #155136

yushangdi · Jun 4, 2025

Summary: fbcode/sigmoid/core/common -> fbcode/caffe2/torch/nativert/common

Torch Native Runtime RFC: pytorch/rfcs#72

Test Plan:

buck run fbcode//mode/dev-nosan  //caffe2/test/cpp/nativert:pytree_test

OSS CI

Rollback Plan:

Differential Revision: D75965059

pytorch-bot · Jun 4, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/155136

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 5a299f0 with merge base 1dd0b1d ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / cuda12.8-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) (gh) (trunk failure)
MISSING REGRESSION TEST

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · Jun 4, 2025

This pull request was exported from Phabricator. Differential Revision: D75965059

Skylion007 · Jun 4, 2025

I'm sure @XuehaiPan has some thoughts here

zhxchen17 · Jun 4, 2025

I think we mean to move the code to nativert/ as part of the impl details. To add some context, we don't intend to expose this as part of public API. Also this pytree impl contains many optimizations we accumulated to high qps low latency model serving use cases and imo we should still recommend ppl to use optree in common cases.

facebook-github-bot · Jun 4, 2025

This pull request was exported from Phabricator. Differential Revision: D75965059

Summary: Pull Request resolved: pytorch#155136 fbcode/sigmoid/core/common -> fbcode/caffe2/torch/nativert/common Test Plan: ``` buck run fbcode//mode/dev-nosan //caffe2/test/cpp/nativert:pytree_test ``` OSS CI Rollback Plan: Differential Revision: D75965059

yushangdi · Jun 4, 2025

Also this pytree impl contains many optimizations we accumulated to high qps low latency model serving use cases and imo we should still recommend ppl to use optree in common cases.

@zhxchen17 sure, I'll move it to nativert/detail instead then.

facebook-github-bot · Jun 4, 2025

This pull request was exported from Phabricator. Differential Revision: D75965059

Summary: Pull Request resolved: pytorch#155136 fbcode/sigmoid/core/common -> fbcode/caffe2/torch/nativert/common Test Plan: ``` buck run fbcode//mode/dev-nosan //caffe2/test/cpp/nativert:pytree_test ``` OSS CI Differential Revision: D75965059

facebook-github-bot · Jun 4, 2025

This pull request was exported from Phabricator. Differential Revision: D75965059

Summary: Pull Request resolved: #155136 fbcode/sigmoid/core/common -> fbcode/caffe2/torch/nativert/common Test Plan: ``` buck run fbcode//mode/dev-nosan //caffe2/test/cpp/nativert:pytree_test ``` OSS CI Differential Revision: D75965059

albanD

@zou3519 how does that align with the other c++ pytree discussion we've been having?

zou3519 · Jun 4, 2025

@zou3519 how does that align with the other c++ pytree discussion we've been having?

@albanD when we say C++ pytree we really mean two things:

Python bindings for C++ functionality to flatten/unflatten PyObject* Python containers
C++ functionality to flatten/unflatten IValues

"C++ Pytree" and optree is (1). We don't have (2) in PyTorch. This PR adds (2).
There are use cases for both, e.g. executorch has both (1) and (2).

My conclusion is that we should allow this PR and that it is unrelated to the optree story. If we're up to it then we should probably name it something else other than "pytree", because there's no Python involved

yushangdi · Jun 4, 2025

If we're up to it then we should probably name it something else other than "pytree", because there's no Python involved

I'd argue it's not completely unrelated to python, because it does register and process the string representation of python types in json, e.g. builtins.tuple, torch.fx.immutable_collections.immutable_dict...

But if we really want a name other than pytree, @zou3519 @zhxchen17 do you have any suggestion for a better name? maybe TreeSpec.h?

torch/nativert/detail/Pytree.h

torch/nativert/detail/Pytree.cpp

yushangdi · Jun 5, 2025

@XuehaiPan @zou3519 How about we just re-use the NestedStructure.h name from https://github.com/pytorch/pytorch/pull? I can change the file name from Pytree.h and Pytree.cpp to NestedStructure.h and NestedStructure.cpp.

…cture (#155136) Summary: Pull Request resolved: #155136 Torch Native Runtime RFC: pytorch/rfcs#72 As part of the effort to open source TorchNativeRuntime (or what we call Sigmoid), we are moving the Pytree implementation to torch/: fbcode/sigmoid/core/common -> fbcode/caffe2/torch/nativert/common We also renamed the functions, the reason behind renaming is in this discussion: #155136 The functionalities and implementation remain the same. Rename the class and functions: 1. `TreeSpec` -> `NestedSpec` 2. `treeSpecLoads` -> `nestedSpecLoads` 3. `treeFlatten` -> `nestedFlatten` 4. `treeUnflatten` -> `nestedUnflatten` 5. `leafApply` -> `ivalueApply` 6. `leafApplyFromArgs` -> `ivalueApplyFromArgs` Test Plan: ``` buck run fbcode//mode/dev-nosan //caffe2/test/cpp/nativert:itree_test ``` OSS CI Reviewed By: zhxchen17 Differential Revision: D75965059

facebook-github-bot · Jun 9, 2025

This pull request was exported from Phabricator. Differential Revision: D75965059

…orch#155136) Summary: Pull Request resolved: pytorch#155136 Torch Native Runtime RFC: pytorch/rfcs#72 As part of the effort to open source TorchNativeRuntime (or what we call Sigmoid), we are moving the Pytree implementation to torch/: fbcode/sigmoid/core/common -> fbcode/caffe2/torch/nativert/common We also renamed the functions, the reason behind renaming is in this discussion: pytorch#155136 The functionalities and implementation remain the same. Rename the class and functions: 1. `TreeSpec` -> `NestedSpec` 2. `treeSpecLoads` -> `nestedSpecLoads` 3. `treeFlatten` -> `nestedFlatten` 4. `treeUnflatten` -> `nestedUnflatten` 5. `leafApply` -> `ivalueApply` 6. `leafApplyFromArgs` -> `ivalueApplyFromArgs` Test Plan: ``` buck run fbcode//mode/dev-nosan //caffe2/test/cpp/nativert:itree_test ``` OSS CI Reviewed By: zhxchen17 Differential Revision: D75965059

yushangdi · Jun 9, 2025

@zou3519 updated to use ITree now :)

facebook-github-bot · Jun 11, 2025

This pull request was exported from Phabricator. Differential Revision: D75965059

…orch#155136) Summary: Pull Request resolved: pytorch#155136 Torch Native Runtime RFC: pytorch/rfcs#72 As part of the effort to open source TorchNativeRuntime (or what we call Sigmoid), we are moving the Pytree implementation to torch/: fbcode/sigmoid/core/common -> fbcode/caffe2/torch/nativert/common We also renamed the functions, the reason behind renaming is in this discussion: pytorch#155136 The functionalities and implementation remain the same. Rename the class and functions: 1. `TreeSpec` -> `NestedSpec` 2. `treeSpecLoads` -> `nestedSpecLoads` 3. `treeFlatten` -> `nestedFlatten` 4. `treeUnflatten` -> `nestedUnflatten` 5. `leafApply` -> `ivalueApply` 6. `leafApplyFromArgs` -> `ivalueApplyFromArgs` Test Plan: ``` buck run fbcode//mode/dev-nosan //caffe2/test/cpp/nativert:itree_test ``` OSS CI Reviewed By: zou3519, zhxchen17 Differential Revision: D75965059

facebook-github-bot · Jun 11, 2025

This pull request was exported from Phabricator. Differential Revision: D75965059

…orch#155136) Summary: Pull Request resolved: pytorch#155136 Torch Native Runtime RFC: pytorch/rfcs#72 As part of the effort to open source TorchNativeRuntime (or what we call Sigmoid), we are moving the Pytree implementation to torch/: fbcode/sigmoid/core/common -> fbcode/caffe2/torch/nativert/common We also renamed the functions, the reason behind renaming is in this discussion: pytorch#155136 The functionalities and implementation remain the same. Rename the class and functions: 1. `TreeSpec` -> `NestedSpec` 2. `treeSpecLoads` -> `nestedSpecLoads` 3. `treeFlatten` -> `nestedFlatten` 4. `treeUnflatten` -> `nestedUnflatten` 5. `leafApply` -> `ivalueApply` 6. `leafApplyFromArgs` -> `ivalueApplyFromArgs` Test Plan: ``` buck run fbcode//mode/dev-nosan //caffe2/test/cpp/nativert:itree_test ``` OSS CI Reviewed By: zou3519, zhxchen17 Differential Revision: D75965059

facebook-github-bot · Jun 11, 2025

This pull request was exported from Phabricator. Differential Revision: D75965059

…orch#155136) Summary: Pull Request resolved: pytorch#155136 Torch Native Runtime RFC: pytorch/rfcs#72 As part of the effort to open source TorchNativeRuntime (or what we call Sigmoid), we are moving the Pytree implementation to torch/: fbcode/sigmoid/core/common -> fbcode/caffe2/torch/nativert/common We also renamed the functions, the reason behind renaming is in this discussion: pytorch#155136 The functionalities and implementation remain the same. Rename the class and functions: 1. `TreeSpec` -> `ITreeSpec` 2. `treeSpecLoads` -> `itreeSpecLoads` 3. `treeFlatten` -> `itreeFlatten` 4. `treeUnflatten` -> `itreeUnflatten` 5. `leafApply` -> `ivalueApply` 6. `leafApplyFromArgs` -> `ivalueApplyFromArgs` Test Plan: ``` buck run fbcode//mode/dev-nosan //caffe2/test/cpp/nativert:itree_test ``` OSS CI Reviewed By: zou3519, zhxchen17 Differential Revision: D75965059

facebook-github-bot · Jun 11, 2025

This pull request was exported from Phabricator. Differential Revision: D75965059

…orch#155136) Summary: Pull Request resolved: pytorch#155136 Torch Native Runtime RFC: pytorch/rfcs#72 As part of the effort to open source TorchNativeRuntime (or what we call Sigmoid), we are moving the Pytree implementation to torch/: fbcode/sigmoid/core/common -> fbcode/caffe2/torch/nativert/common We also renamed the functions, the reason behind renaming is in this discussion: pytorch#155136 The functionalities and implementation remain the same. Rename the class and functions: 1. `TreeSpec` -> `ITreeSpec` 2. `treeSpecLoads` -> `itreeSpecLoads` 3. `treeFlatten` -> `itreeFlatten` 4. `treeUnflatten` -> `itreeUnflatten` 5. `leafApply` -> `ivalueApply` 6. `leafApplyFromArgs` -> `ivalueApplyFromArgs` Test Plan: ``` buck run fbcode//mode/dev-nosan //caffe2/test/cpp/nativert:itree_test ``` OSS CI Reviewed By: zou3519, zhxchen17 Differential Revision: D75965059

facebook-github-bot · Jun 11, 2025

This pull request was exported from Phabricator. Differential Revision: D75965059

facebook-github-bot · Jun 12, 2025

@pytorchbot merge

(Initiating merge automatically since Phabricator Diff has merged)

pytorchmergebot · Jun 12, 2025

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Summary: fbcode/sigmoid/core/common -> fbcode/caffe2/torch/nativert/common Torch Native Runtime RFC: pytorch/rfcs#72 Test Plan: ``` buck run fbcode//mode/dev-nosan //caffe2/test/cpp/nativert:pytree_test ``` OSS CI Rollback Plan: Differential Revision: D75965059 Pull Request resolved: pytorch#155136 Approved by: https://github.com/zhxchen17, https://github.com/XuehaiPan, https://github.com/zou3519

facebook-github-bot added the fb-exported label Jun 4, 2025

Skylion007 requested a review from XuehaiPan June 4, 2025 17:29

Skylion007 self-assigned this Jun 4, 2025

zhxchen17 added the topic: not user facing topic category label Jun 4, 2025

yushangdi force-pushed the export-D75965059 branch from 0f82827 to 05666b3 Compare June 4, 2025 17:45

yushangdi force-pushed the export-D75965059 branch from 05666b3 to 08244e4 Compare June 4, 2025 17:57

yushangdi force-pushed the export-D75965059 branch from 08244e4 to 601cb83 Compare June 4, 2025 18:51

zhxchen17 approved these changes Jun 4, 2025

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jun 4, 2025

yushangdi requested review from albanD and cyyever June 4, 2025 18:58

albanD previously requested changes Jun 4, 2025

View reviewed changes

yushangdi requested a review from zou3519 June 4, 2025 19:50

yushangdi requested a review from albanD June 4, 2025 23:00

XuehaiPan reviewed Jun 5, 2025

View reviewed changes

torch/nativert/detail/Pytree.h Outdated Show resolved Hide resolved

XuehaiPan reviewed Jun 5, 2025

View reviewed changes

torch/nativert/detail/Pytree.cpp Outdated Show resolved Hide resolved

yushangdi requested a review from zou3519 June 9, 2025 21:47

yushangdi force-pushed the export-D75965059 branch from 75d382e to 1522644 Compare June 9, 2025 22:03

zou3519 approved these changes Jun 11, 2025

View reviewed changes

yushangdi force-pushed the export-D75965059 branch from 1522644 to 9767a8b Compare June 11, 2025 16:58

yushangdi force-pushed the export-D75965059 branch from 9767a8b to 93e11c0 Compare June 11, 2025 17:06

yushangdi force-pushed the export-D75965059 branch from 93e11c0 to f06e5e8 Compare June 11, 2025 19:20

yushangdi force-pushed the export-D75965059 branch from f06e5e8 to bec2859 Compare June 11, 2025 19:52

yushangdi force-pushed the export-D75965059 branch from bec2859 to 5a299f0 Compare June 11, 2025 20:05

pytorchmergebot added the merging label Jun 12, 2025

pytorchmergebot closed this in 4e19477 Jun 12, 2025

pytorchmergebot added Merged and removed merging labels Jun 12, 2025

Search code, repositories, users, issues, pull requests...

[nativert] Move Pytree #155136

[nativert] Move Pytree #155136

Uh oh!

Conversation

yushangdi commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/155136

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

facebook-github-bot commented Jun 4, 2025

Uh oh!

Skylion007 commented Jun 4, 2025

Uh oh!

zhxchen17 commented Jun 4, 2025

Uh oh!

facebook-github-bot commented Jun 4, 2025

Uh oh!

yushangdi commented Jun 4, 2025

Uh oh!

facebook-github-bot commented Jun 4, 2025

Uh oh!

facebook-github-bot commented Jun 4, 2025

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

zou3519 commented Jun 4, 2025

Uh oh!

yushangdi commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yushangdi commented Jun 5, 2025

Uh oh!

facebook-github-bot commented Jun 9, 2025

Uh oh!

yushangdi commented Jun 9, 2025

Uh oh!

facebook-github-bot commented Jun 11, 2025

Uh oh!

facebook-github-bot commented Jun 11, 2025

Uh oh!

facebook-github-bot commented Jun 11, 2025

Uh oh!

facebook-github-bot commented Jun 11, 2025

Uh oh!

facebook-github-bot commented Jun 11, 2025

Uh oh!

facebook-github-bot commented Jun 12, 2025

Uh oh!

pytorchmergebot commented Jun 12, 2025

Merge started

Uh oh!

Uh oh!

yushangdi commented Jun 4, 2025 •

edited

Loading

pytorch-bot bot commented Jun 4, 2025 •

edited

Loading

yushangdi commented Jun 4, 2025 •

edited

Loading