Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit f4fde65

Browse filesBrowse files
authored
fix: spill module mapping args to a file (#2644)
Calls to the modules mapping rule contains very long command line args due to the use of the full `wheels` parameter. This change adds support for spilling the args into a file as needed. In addition, it improves the performance of the `modules_mapping` rule: * Remove the calls `to_list` that are unnecessary on the depset. * Remove the iteration over the depset when passing to `args`, and other calls to `.path`, and instead let args do this lazily.
1 parent c7aa989 commit f4fde65
Copy full SHA for f4fde65

File tree

Expand file treeCollapse file tree

3 files changed

+16
-4
lines changed
Filter options
Expand file treeCollapse file tree

3 files changed

+16
-4
lines changed

‎CHANGELOG.md

Copy file name to clipboardExpand all lines: CHANGELOG.md
+2Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,8 @@ Unreleased changes template.
6363
is now the default. Note that running as root may still cause spurious
6464
Bazel cache invalidation
6565
([#1169](https://github.com/bazelbuild/rules_python/issues/1169)).
66+
* (gazelle) Don't collapse depsets to a list or into args when generating the modules mapping file.
67+
Support spilling modules mapping args into a params file.
6668

6769
{#v0-0-0-added}
6870
### Added

‎gazelle/modules_mapping/def.bzl

Copy file name to clipboardExpand all lines: gazelle/modules_mapping/def.bzl
+11-4Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,18 +25,25 @@ module name doesn't match the wheel distribution name.
2525

2626
def _modules_mapping_impl(ctx):
2727
modules_mapping = ctx.actions.declare_file(ctx.attr.modules_mapping_name)
28-
args = ctx.actions.args()
2928
all_wheels = depset(
3029
[whl for whl in ctx.files.wheels],
3130
transitive = [dep[DefaultInfo].files for dep in ctx.attr.wheels] + [dep[DefaultInfo].data_runfiles.files for dep in ctx.attr.wheels],
3231
)
33-
args.add("--output_file", modules_mapping.path)
32+
33+
args = ctx.actions.args()
34+
35+
# Spill parameters to a file prefixed with '@'. Note, the '@' prefix is the same
36+
# prefix as used in the `generator.py` in `fromfile_prefix_chars` attribute.
37+
args.use_param_file(param_file_arg = "@%s")
38+
args.set_param_file_format(format = "multiline")
3439
if ctx.attr.include_stub_packages:
3540
args.add("--include_stub_packages")
41+
args.add("--output_file", modules_mapping)
3642
args.add_all("--exclude_patterns", ctx.attr.exclude_patterns)
37-
args.add_all("--wheels", [whl.path for whl in all_wheels.to_list()])
43+
args.add_all("--wheels", all_wheels)
44+
3845
ctx.actions.run(
39-
inputs = all_wheels.to_list(),
46+
inputs = all_wheels,
4047
outputs = [modules_mapping],
4148
executable = ctx.executable._generator,
4249
arguments = [args],

‎gazelle/modules_mapping/generator.py

Copy file name to clipboardExpand all lines: gazelle/modules_mapping/generator.py
+3Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,9 @@ def data_has_purelib_or_platlib(path):
152152
parser = argparse.ArgumentParser(
153153
prog="generator",
154154
description="Generates the modules mapping used by the Gazelle manifest.",
155+
# Automatically read parameters from a file. Note, the '@' is the same prefix
156+
# as set in the 'args.use_param_file' in the bazel rule.
157+
fromfile_prefix_chars="@",
155158
)
156159
parser.add_argument("--output_file", type=str)
157160
parser.add_argument("--include_stub_packages", action="store_true")

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.