Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit cc43594

Browse filesBrowse files
rickeylevrickeylev
authored
cleanup: Typos, doc, and formatting cleanup in pip extension et al (bazel-contrib#1275)
* Corrects some typos in docs * Expands the user-facing documentation * Fixes errors having newlines in the middle of them * Renames some internal functions to be more self-describing.
1 parent 1a333ce commit cc43594
Copy full SHA for cc43594

File tree

Expand file treeCollapse file tree

4 files changed

+92
-50
lines changed
Filter options
Expand file treeCollapse file tree

4 files changed

+92
-50
lines changed

‎docs/pip_repository.md

Copy file name to clipboardExpand all lines: docs/pip_repository.md
+1-1Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎examples/bzlmod/MODULE.bazel

Copy file name to clipboardExpand all lines: examples/bzlmod/MODULE.bazel
+9-7Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -34,16 +34,17 @@ python.toolchain(
3434
# You only need to load this repositories if you are using multiple Python versions.
3535
# See the tests folder for various examples on using multiple Python versions.
3636
# The names "python_3_9" and "python_3_10" are autmatically created by the repo
37-
# rules based on the python_versions.
37+
# rules based on the `python_version` arg values.
3838
use_repo(python, "python_3_10", "python_3_9", "python_aliases")
3939

4040
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
4141

42-
# For each pip setup we call pip.parse. We can pass in various options
43-
# but typically we are passing in a requirements and an interpreter.
44-
# If you provide the python_version we will attempt to determine
45-
# the interpreter target automatically. Otherwise use python_interpreter_target
46-
# to override the lookup.
42+
# To fetch pip dependencies, use pip.parse. We can pass in various options,
43+
# but typically we pass requirements and the Python version. The Python
44+
# version must have been configured by a corresponding `python.toolchain()`
45+
# call.
46+
# Alternatively, `python_interpreter_target` can be used to directly specify
47+
# the Python interpreter to run to resolve dependencies.
4748
pip.parse(
4849
hub_name = "pip",
4950
python_version = "3.9",
@@ -57,7 +58,8 @@ pip.parse(
5758
requirements_windows = "//:requirements_windows_3_10.txt",
5859
)
5960

60-
# we use the pip_39 repo because entry points are not yet supported.
61+
# NOTE: The pip_39 repo is only used because the plain `@pip` repo doesn't
62+
# yet support entry points; see https://github.com/bazelbuild/rules_python/issues/1262
6163
use_repo(pip, "pip", "pip_39")
6264

6365
bazel_dep(name = "other_module", version = "", repo_name = "our_other_module")

‎python/extensions/pip.bzl

Copy file name to clipboardExpand all lines: python/extensions/pip.bzl
+81-41Lines changed: 81 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -27,21 +27,25 @@ load(
2727
)
2828
load("@rules_python//python/pip_install:requirements_parser.bzl", parse_requirements = "parse")
2929

30-
def _create_pip(module_ctx, pip_attr, whl_map):
30+
def _create_versioned_pip_and_whl_repos(module_ctx, pip_attr, whl_map):
3131
python_interpreter_target = pip_attr.python_interpreter_target
3232

3333
# if we do not have the python_interpreter set in the attributes
3434
# we programtically find it.
35+
hub_name = pip_attr.hub_name
3536
if python_interpreter_target == None:
3637
python_name = "python_{}".format(pip_attr.python_version.replace(".", "_"))
3738
if python_name not in INTERPRETER_LABELS.keys():
38-
fail("""
39-
Unable to find '{}' in the list of interpreters please update your pip.parse call with the correct python name
40-
""".format(pip_attr.python_name))
41-
39+
fail((
40+
"Unable to find interpreter for pip hub '{hub_name}' for " +
41+
"python_version={version}: Make sure a corresponding " +
42+
'`python.toolchain(python_version="{version}")` call exists'
43+
).format(
44+
hub_name = hub_name,
45+
version = pip_attr.python_version,
46+
))
4247
python_interpreter_target = INTERPRETER_LABELS[python_name]
4348

44-
hub_name = pip_attr.hub_name
4549
pip_name = hub_name + "_{}".format(pip_attr.python_version.replace(".", ""))
4650
requrements_lock = locked_requirements_label(module_ctx, pip_attr)
4751

@@ -93,10 +97,10 @@ Unable to find '{}' in the list of interpreters please update your pip.parse cal
9397
whl_map[hub_name][whl_name][pip_attr.python_version] = pip_name + "_"
9498

9599
def _pip_impl(module_ctx):
96-
"""Implmentation of a class tag that creates the pip hub(s) and corresponding pip spoke, alias and whl repositories.
100+
"""Implementation of a class tag that creates the pip hub(s) and corresponding pip spoke, alias and whl repositories.
97101
98-
This implmentation iterates through all of the "pip.parse" calls and creates
99-
different pip hubs repositories based on the "hub_name". Each of the
102+
This implmentation iterates through all of the `pip.parse` calls and creates
103+
different pip hub repositories based on the "hub_name". Each of the
100104
pip calls create spoke repos that uses a specific Python interpreter.
101105
102106
In a MODULES.bazel file we have:
@@ -115,13 +119,13 @@ def _pip_impl(module_ctx):
115119
)
116120
117121
118-
For instance we have a hub with the name of "pip".
122+
For instance, we have a hub with the name of "pip".
119123
A repository named the following is created. It is actually called last when
120124
all of the pip spokes are collected.
121125
122126
- @@rules_python~override~pip~pip
123127
124-
As show in the example code above we have the following.
128+
As shown in the example code above we have the following.
125129
Two different pip.parse statements exist in MODULE.bazel provide the hub_name "pip".
126130
These definitions create two different pip spoke repositories that are
127131
related to the hub "pip".
@@ -185,22 +189,32 @@ def _pip_impl(module_ctx):
185189

186190
for mod in module_ctx.modules:
187191
for pip_attr in mod.tags.parse:
188-
if pip_attr.hub_name in pip_hub_map:
192+
hub_name = pip_attr.hub_name
193+
if hub_name in pip_hub_map:
189194
# We cannot have two hubs with the same name in different
190195
# modules.
191-
if pip_hub_map[pip_attr.hub_name].module_name != mod.name:
192-
fail("""Unable to create pip with the hub_name '{}', same hub name
193-
in a different module found.""".format(pip_attr.hub_name))
194-
195-
if pip_attr.python_version in pip_hub_map[pip_attr.hub_name].python_versions:
196-
fail(
197-
"""Unable to create pip with the hub_name '{}', same hub name
198-
using the same Python repo name '{}' found in module '{}'.""".format(
199-
pip_attr.hub_name,
200-
pip_attr.python_version,
201-
mod.name,
202-
),
203-
)
196+
if pip_hub_map[hub_name].module_name != mod.name:
197+
fail((
198+
"Duplicate cross-module pip hub named '{hub}': pip hub " +
199+
"names must be unique across modules. First defined " +
200+
"by module '{first_module}', second attempted by " +
201+
"module '{second_module}'"
202+
).format(
203+
hub = hub_name,
204+
first_module = pip_hub_map[hub_name].module_name,
205+
second_module = mod.name,
206+
))
207+
208+
if pip_attr.python_version in pip_hub_map[hub_name].python_versions:
209+
fail((
210+
"Duplicate pip python version '{version}' for hub " +
211+
"'{hub}' in module '{module}': the Python versions " +
212+
"used for a hub must be unique"
213+
).format(
214+
hub = hub_name,
215+
module = mod.name,
216+
version = pip_attr.python_version,
217+
))
204218
else:
205219
pip_hub_map[pip_attr.hub_name].python_versions.append(pip_attr.python_version)
206220
else:
@@ -209,17 +223,19 @@ def _pip_impl(module_ctx):
209223
python_versions = [pip_attr.python_version],
210224
)
211225

212-
_create_pip(module_ctx, pip_attr, hub_whl_map)
226+
_create_versioned_pip_and_whl_repos(module_ctx, pip_attr, hub_whl_map)
213227

214228
for hub_name, whl_map in hub_whl_map.items():
215229
for whl_name, version_map in whl_map.items():
216230
if DEFAULT_PYTHON_VERSION not in version_map:
217-
fail(
218-
"""
219-
Unable to find the default python version in the version map, please update your requirements files
220-
to include Python '{}'.
221-
""".format(DEFAULT_PYTHON_VERSION),
222-
)
231+
fail((
232+
"Default python version '{version}' missing in pip " +
233+
"hub '{hub}': update your pip.parse() calls so that " +
234+
'includes `python_version = "{version}"`'
235+
).format(
236+
version = DEFAULT_PYTHON_VERSION,
237+
hub = hub_name,
238+
))
223239

224240
# Create the alias repositories which contains different select
225241
# statements These select statements point to the different pip
@@ -247,14 +263,34 @@ def _pip_parse_ext_attrs():
247263
"hub_name": attr.string(
248264
mandatory = True,
249265
doc = """
250-
The unique hub name. Mulitple pip.parse calls that contain the same hub name,
251-
create spokes for specific Python versions.
266+
The name of the repo pip dependencies will be accessible from.
267+
268+
This name must be unique between modules; unless your module is guaranteed to
269+
always be the root module, it's highly recommended to include your module name
270+
in the hub name. Repo mapping, `use_repo(..., pip="my_modules_pip_deps")`, can
271+
be used for shorter local names within your module.
272+
273+
Within a module, the same `hub_name` can be specified to group different Python
274+
versions of pip dependencies under one repository name. This allows using a
275+
Python version-agnostic name when referring to pip dependencies; the
276+
correct version will be automatically selected.
277+
278+
Typically, a module will only have a single hub of pip dependencies, but this
279+
is not required. Each hub is a separate resolution of pip dependencies. This
280+
means if different programs need different versions of some library, separate
281+
hubs can be created, and each program can use its respective hub's targets.
282+
Targets from different hubs should not be used together.
252283
""",
253284
),
254285
"python_version": attr.string(
255286
mandatory = True,
256287
doc = """
257-
The Python version for the pip spoke.
288+
The Python version to use for resolving the pip dependencies. If not specified,
289+
then the default Python version (as set by the root module or rules_python)
290+
will be used.
291+
292+
The version specified here must have a corresponding `python.toolchain()`
293+
configured.
258294
""",
259295
),
260296
}, **pip_repository_attrs)
@@ -270,12 +306,16 @@ The Python version for the pip spoke.
270306

271307
pip = module_extension(
272308
doc = """\
273-
This extension is used to create a pip hub and all of the spokes that are part of that hub.
274-
We can have multiple different hubs, but we cannot have hubs that have the same name in
275-
different modules. Each hub needs one or more spokes. A spoke contains a specific version
276-
of Python, and the requirement(s) files that are unquie to that Python version.
277-
In order to add more spokes you call this extension mulitiple times using the same hub
278-
name.
309+
This extension is used to make dependencies from pip available.
310+
311+
To use, call `pip.parse()` and specify `hub_name` and your requirements file.
312+
Dependencies will be downloaded and made available in a repo named after the
313+
`hub_name` argument.
314+
315+
Each `pip.parse()` call configures a particular Python version. Multiple calls
316+
can be made to configure different Python versions, and will be grouped by
317+
the `hub_name` argument. This allows the same logical name, e.g. `@pip//numpy`
318+
to automatically resolve to different, Python version-specific, libraries.
279319
""",
280320
implementation = _pip_impl,
281321
tag_classes = {

‎python/pip_install/pip_repository.bzl

Copy file name to clipboardExpand all lines: python/pip_install/pip_repository.bzl
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -363,7 +363,7 @@ pip_hub_repository_bzlmod_attrs = {
363363

364364
pip_hub_repository_bzlmod = repository_rule(
365365
attrs = pip_hub_repository_bzlmod_attrs,
366-
doc = """A rule for bzlmod mulitple pip repository creation. Intended for private use only.""",
366+
doc = """A rule for bzlmod mulitple pip repository creation. PRIVATE USE ONLY.""",
367367
implementation = _pip_hub_repository_bzlmod_impl,
368368
)
369369

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.