Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit f6c0e46

Browse filesBrowse files
alexeaglealexeagle
committed
Add pip_repository documentation
The macros are leaky and otherwise you have to read sources to find out about the kwargs
1 parent f480ebe commit f6c0e46
Copy full SHA for f6c0e46

File tree

Expand file treeCollapse file tree

5 files changed

+169
-11
lines changed
Filter options
Expand file treeCollapse file tree

5 files changed

+169
-11
lines changed

‎docs/BUILD

Copy file name to clipboardExpand all lines: docs/BUILD
+13Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ licenses(["notice"]) # Apache 2.0
2424
_DOCS = {
2525
"packaging": "//docs:packaging-docs",
2626
"pip": "//docs:pip-docs",
27+
"pip_repository": "//docs:pip-repository",
2728
"python": "//docs:core-docs",
2829
}
2930

@@ -104,6 +105,18 @@ stardoc(
104105
],
105106
)
106107

108+
stardoc(
109+
name = "pip-repository",
110+
out = "pip_repository.md_",
111+
input = "//python/pip_install:pip_repository.bzl",
112+
target_compatible_with = _NOT_WINDOWS,
113+
deps = [
114+
":bazel_repo_tools",
115+
":pip_install_bzl",
116+
"//third_party/github.com/bazelbuild/bazel-skylib/lib:versions",
117+
],
118+
)
119+
107120
stardoc(
108121
name = "packaging-docs",
109122
out = "packaging.md_",

‎docs/pip.md

Copy file name to clipboardExpand all lines: docs/pip.md
+7-5Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,8 @@ Accepts a `requirements.txt` file and installs the dependencies listed within.
7474

7575
Those dependencies become available in a generated `requirements.bzl` file.
7676

77-
This macro runs a repository rule that invokes `pip`. In your WORKSPACE file:
77+
This macro wraps the [`pip_repository`](./pip_repository.md) rule that invokes `pip`.
78+
In your WORKSPACE file:
7879

7980
```python
8081
pip_install(
@@ -142,7 +143,7 @@ alias(
142143
| :-------------: | :-------------: | :-------------: |
143144
| requirements | A 'requirements.txt' pip requirements file. | <code>None</code> |
144145
| name | A unique name for the created external repository (default 'pip'). | <code>"pip"</code> |
145-
| kwargs | Keyword arguments passed directly to the <code>pip_repository</code> repository rule. | none |
146+
| kwargs | Additional arguments to the [<code>pip_repository</code>](./pip_repository.md) repository rule. | none |
146147

147148

148149
<a name="#pip_parse"></a>
@@ -158,7 +159,8 @@ Accepts a locked/compiled requirements file and installs the dependencies listed
158159
Those dependencies become available in a generated `requirements.bzl` file.
159160
You can instead check this `requirements.bzl` file into your repo, see the "vendoring" section below.
160161

161-
This macro runs a repository rule that invokes `pip`. In your WORKSPACE file:
162+
This macro wraps the [`pip_repository`](./pip_repository.md) rule that invokes `pip`, with `incremental` set.
163+
In your WORKSPACE file:
162164

163165
```python
164166
load("@rules_python//python:pip.bzl", "pip_parse")
@@ -242,9 +244,9 @@ See the example in rules_python/examples/pip_parse_vendored.
242244

243245
| Name | Description | Default Value |
244246
| :-------------: | :-------------: | :-------------: |
245-
| requirements_lock | A fully resolved 'requirements.txt' pip requirement file containing the transitive set of your dependencies. If this file is passed instead of 'requirements' no resolve will take place and pip_repository will create individual repositories for each of your dependencies so that wheels are fetched/built only for the targets specified by 'build/run/test'. | none |
247+
| requirements_lock | A fully resolved 'requirements.txt' pip requirement file containing the transitive set of your dependencies. If this file is passed instead of 'requirements' no resolve will take place and pip_repository will create individual repositories for each of your dependencies so that wheels are fetched/built only for the targets specified by 'build/run/test'. Note that if your lockfile is platform-dependent, you can use the <code>requirements_[platform]</code> attributes. | none |
246248
| name | The name of the generated repository. The generated repositories containing each requirement will be of the form &lt;name&gt;_&lt;requirement-name&gt;. | <code>"pip_parsed_deps"</code> |
247-
| kwargs | Additional keyword arguments for the underlying <code>pip_repository</code> rule. | none |
249+
| kwargs | Additional arguments to the [<code>pip_repository</code>](./pip_repository.md) repository rule. | none |
248250

249251

250252
<a name="#pip_repositories"></a>

‎docs/pip_repository.md

Copy file name to clipboard
+140Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
<!-- Generated with Stardoc: http://skydoc.bazel.build -->
2+
3+
<a name="#pip_repository"></a>
4+
5+
## pip_repository
6+
7+
<pre>
8+
pip_repository(<a href="#pip_repository-name">name</a>, <a href="#pip_repository-annotations">annotations</a>, <a href="#pip_repository-enable_implicit_namespace_pkgs">enable_implicit_namespace_pkgs</a>, <a href="#pip_repository-environment">environment</a>, <a href="#pip_repository-extra_pip_args">extra_pip_args</a>,
9+
<a href="#pip_repository-incremental">incremental</a>, <a href="#pip_repository-isolated">isolated</a>, <a href="#pip_repository-pip_data_exclude">pip_data_exclude</a>, <a href="#pip_repository-python_interpreter">python_interpreter</a>, <a href="#pip_repository-python_interpreter_target">python_interpreter_target</a>,
10+
<a href="#pip_repository-quiet">quiet</a>, <a href="#pip_repository-repo_prefix">repo_prefix</a>, <a href="#pip_repository-requirements">requirements</a>, <a href="#pip_repository-requirements_darwin">requirements_darwin</a>, <a href="#pip_repository-requirements_linux">requirements_linux</a>,
11+
<a href="#pip_repository-requirements_lock">requirements_lock</a>, <a href="#pip_repository-requirements_windows">requirements_windows</a>, <a href="#pip_repository-timeout">timeout</a>)
12+
</pre>
13+
14+
A rule for importing `requirements.txt` dependencies into Bazel.
15+
16+
This rule imports a `requirements.txt` file and generates a new
17+
`requirements.bzl` file. This is used via the `WORKSPACE` pattern:
18+
19+
```python
20+
pip_repository(
21+
name = "foo",
22+
requirements = ":requirements.txt",
23+
)
24+
```
25+
26+
You can then reference imported dependencies from your `BUILD` file with:
27+
28+
```python
29+
load("@foo//:requirements.bzl", "requirement")
30+
py_library(
31+
name = "bar",
32+
...
33+
deps = [
34+
"//my/other:dep",
35+
requirement("requests"),
36+
requirement("numpy"),
37+
],
38+
)
39+
```
40+
41+
Or alternatively:
42+
```python
43+
load("@foo//:requirements.bzl", "all_requirements")
44+
py_binary(
45+
name = "baz",
46+
...
47+
deps = [
48+
":foo",
49+
] + all_requirements,
50+
)
51+
```
52+
53+
54+
**ATTRIBUTES**
55+
56+
57+
| Name | Description | Type | Mandatory | Default |
58+
| :-------------: | :-------------: | :-------------: | :-------------: | :-------------: |
59+
| name | A unique name for this repository. | <a href="https://bazel.build/docs/build-ref.html#name">Name</a> | required | |
60+
| annotations | Optional annotations to apply to packages | <a href="https://bazel.build/docs/skylark/lib/dict.html">Dictionary: String -> String</a> | optional | {} |
61+
| enable_implicit_namespace_pkgs | If true, disables conversion of native namespace packages into pkg-util style namespace packages. When set all py_binary and py_test targets must specify either <code>legacy_create_init=False</code> or the global Bazel option <code>--incompatible_default_to_explicit_init_py</code> to prevent <code>__init__.py</code> being automatically generated in every directory.<br><br>This option is required to support some packages which cannot handle the conversion to pkg-util style. | Boolean | optional | False |
62+
| environment | Environment variables to set in the pip subprocess. Can be used to set common variables such as <code>http_proxy</code>, <code>https_proxy</code> and <code>no_proxy</code> Note that pip is run with "--isolated" on the CLI so PIP_&lt;VAR&gt;_&lt;NAME&gt; style env vars are ignored, but env vars that control requests and urllib3 can be passed. | <a href="https://bazel.build/docs/skylark/lib/dict.html">Dictionary: String -> String</a> | optional | {} |
63+
| extra_pip_args | Extra arguments to pass on to pip. Must not contain spaces. | List of strings | optional | [] |
64+
| incremental | Create the repository in incremental mode. | Boolean | optional | False |
65+
| isolated | Whether or not to pass the [--isolated](https://pip.pypa.io/en/stable/cli/pip/#cmdoption-isolated) flag to the underlying pip command. Alternatively, the <code>RULES_PYTHON_PIP_ISOLATED</code> enviornment varaible can be used to control this flag. | Boolean | optional | True |
66+
| pip_data_exclude | Additional data exclusion parameters to add to the pip packages BUILD file. | List of strings | optional | [] |
67+
| python_interpreter | The python interpreter to use. This can either be an absolute path or the name of a binary found on the host's <code>PATH</code> environment variable. If no value is set <code>python3</code> is defaulted for Unix systems and <code>python.exe</code> for Windows. | String | optional | "" |
68+
| python_interpreter_target | If you are using a custom python interpreter built by another repository rule, use this attribute to specify its BUILD target. This allows pip_repository to invoke pip using the same interpreter as your toolchain. If set, takes precedence over python_interpreter. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
69+
| quiet | If True, suppress printing stdout and stderr output to the terminal. | Boolean | optional | True |
70+
| repo_prefix | Prefix for the generated packages. For non-incremental mode the packages will be of the form<br><br>@&lt;name&gt;//&lt;prefix&gt;&lt;sanitized-package-name&gt;/...<br><br>For incremental mode the packages will be of the form<br><br>@&lt;prefix&gt;&lt;sanitized-package-name&gt;//... | String | optional | "" |
71+
| requirements | A 'requirements.txt' pip requirements file. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
72+
| requirements_darwin | Override the requirements_lock attribute when the host platform is Mac OS | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
73+
| requirements_linux | Override the requirements_lock attribute when the host platform is Linux | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
74+
| requirements_lock | A fully resolved 'requirements.txt' pip requirement file containing the transitive set of your dependencies. If this file is passed instead of 'requirements' no resolve will take place and pip_repository will create individual repositories for each of your dependencies so that wheels are fetched/built only for the targets specified by 'build/run/test'. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
75+
| requirements_windows | Override the requirements_lock attribute when the host platform is Windows | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
76+
| timeout | Timeout (in seconds) on the rule's execution duration. | Integer | optional | 600 |
77+
78+
79+
<a name="#whl_library"></a>
80+
81+
## whl_library
82+
83+
<pre>
84+
whl_library(<a href="#whl_library-name">name</a>, <a href="#whl_library-annotation">annotation</a>, <a href="#whl_library-enable_implicit_namespace_pkgs">enable_implicit_namespace_pkgs</a>, <a href="#whl_library-environment">environment</a>, <a href="#whl_library-extra_pip_args">extra_pip_args</a>, <a href="#whl_library-isolated">isolated</a>,
85+
<a href="#whl_library-pip_data_exclude">pip_data_exclude</a>, <a href="#whl_library-python_interpreter">python_interpreter</a>, <a href="#whl_library-python_interpreter_target">python_interpreter_target</a>, <a href="#whl_library-quiet">quiet</a>, <a href="#whl_library-repo">repo</a>, <a href="#whl_library-repo_prefix">repo_prefix</a>,
86+
<a href="#whl_library-requirement">requirement</a>, <a href="#whl_library-timeout">timeout</a>)
87+
</pre>
88+
89+
90+
Download and extracts a single wheel based into a bazel repo based on the requirement string passed in.
91+
Instantiated from pip_repository and inherits config options from there.
92+
93+
**ATTRIBUTES**
94+
95+
96+
| Name | Description | Type | Mandatory | Default |
97+
| :-------------: | :-------------: | :-------------: | :-------------: | :-------------: |
98+
| name | A unique name for this repository. | <a href="https://bazel.build/docs/build-ref.html#name">Name</a> | required | |
99+
| annotation | Optional json encoded file containing annotation to apply to the extracted wheel. See <code>package_annotation</code> | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
100+
| enable_implicit_namespace_pkgs | If true, disables conversion of native namespace packages into pkg-util style namespace packages. When set all py_binary and py_test targets must specify either <code>legacy_create_init=False</code> or the global Bazel option <code>--incompatible_default_to_explicit_init_py</code> to prevent <code>__init__.py</code> being automatically generated in every directory.<br><br>This option is required to support some packages which cannot handle the conversion to pkg-util style. | Boolean | optional | False |
101+
| environment | Environment variables to set in the pip subprocess. Can be used to set common variables such as <code>http_proxy</code>, <code>https_proxy</code> and <code>no_proxy</code> Note that pip is run with "--isolated" on the CLI so PIP_&lt;VAR&gt;_&lt;NAME&gt; style env vars are ignored, but env vars that control requests and urllib3 can be passed. | <a href="https://bazel.build/docs/skylark/lib/dict.html">Dictionary: String -> String</a> | optional | {} |
102+
| extra_pip_args | Extra arguments to pass on to pip. Must not contain spaces. | List of strings | optional | [] |
103+
| isolated | Whether or not to pass the [--isolated](https://pip.pypa.io/en/stable/cli/pip/#cmdoption-isolated) flag to the underlying pip command. Alternatively, the <code>RULES_PYTHON_PIP_ISOLATED</code> enviornment varaible can be used to control this flag. | Boolean | optional | True |
104+
| pip_data_exclude | Additional data exclusion parameters to add to the pip packages BUILD file. | List of strings | optional | [] |
105+
| python_interpreter | The python interpreter to use. This can either be an absolute path or the name of a binary found on the host's <code>PATH</code> environment variable. If no value is set <code>python3</code> is defaulted for Unix systems and <code>python.exe</code> for Windows. | String | optional | "" |
106+
| python_interpreter_target | If you are using a custom python interpreter built by another repository rule, use this attribute to specify its BUILD target. This allows pip_repository to invoke pip using the same interpreter as your toolchain. If set, takes precedence over python_interpreter. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | None |
107+
| quiet | If True, suppress printing stdout and stderr output to the terminal. | Boolean | optional | True |
108+
| repo | Pointer to parent repo name. Used to make these rules rerun if the parent repo changes. | String | required | |
109+
| repo_prefix | Prefix for the generated packages. For non-incremental mode the packages will be of the form<br><br>@&lt;name&gt;//&lt;prefix&gt;&lt;sanitized-package-name&gt;/...<br><br>For incremental mode the packages will be of the form<br><br>@&lt;prefix&gt;&lt;sanitized-package-name&gt;//... | String | optional | "" |
110+
| requirement | Python requirement string describing the package to make available | String | required | |
111+
| timeout | Timeout (in seconds) on the rule's execution duration. | Integer | optional | 600 |
112+
113+
114+
<a name="#package_annotation"></a>
115+
116+
## package_annotation
117+
118+
<pre>
119+
package_annotation(<a href="#package_annotation-additive_build_content">additive_build_content</a>, <a href="#package_annotation-copy_files">copy_files</a>, <a href="#package_annotation-copy_executables">copy_executables</a>, <a href="#package_annotation-data">data</a>, <a href="#package_annotation-data_exclude_glob">data_exclude_glob</a>,
120+
<a href="#package_annotation-srcs_exclude_glob">srcs_exclude_glob</a>)
121+
</pre>
122+
123+
Annotations to apply to the BUILD file content from package generated from a `pip_repository` rule.
124+
125+
[cf]: https://github.com/bazelbuild/bazel-skylib/blob/main/docs/copy_file_doc.md
126+
127+
128+
**PARAMETERS**
129+
130+
131+
| Name | Description | Default Value |
132+
| :-------------: | :-------------: | :-------------: |
133+
| additive_build_content | Raw text to add to the generated <code>BUILD</code> file of a package. | <code>None</code> |
134+
| copy_files | A mapping of <code>src</code> and <code>out</code> files for [@bazel_skylib//rules:copy_file.bzl][cf] | <code>{}</code> |
135+
| copy_executables | A mapping of <code>src</code> and <code>out</code> files for [@bazel_skylib//rules:copy_file.bzl][cf]. Targets generated here will also be flagged as executable. | <code>{}</code> |
136+
| data | A list of labels to add as <code>data</code> dependencies to the generated <code>py_library</code> target. | <code>[]</code> |
137+
| data_exclude_glob | A list of exclude glob patterns to add as <code>data</code> to the generated <code>py_library</code> target. | <code>[]</code> |
138+
| srcs_exclude_glob | A list of labels to add as <code>srcs</code> to the generated <code>py_library</code> target. | <code>[]</code> |
139+
140+

‎python/pip.bzl

Copy file name to clipboardExpand all lines: python/pip.bzl
+8-5Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,8 @@ def pip_install(requirements = None, name = "pip", **kwargs):
2525
2626
Those dependencies become available in a generated `requirements.bzl` file.
2727
28-
This macro runs a repository rule that invokes `pip`. In your WORKSPACE file:
28+
This macro wraps the [`pip_repository`](./pip_repository.md) rule that invokes `pip`.
29+
In your WORKSPACE file:
2930
3031
```python
3132
pip_install(
@@ -88,7 +89,7 @@ def pip_install(requirements = None, name = "pip", **kwargs):
8889
Args:
8990
requirements (Label): A 'requirements.txt' pip requirements file.
9091
name (str, optional): A unique name for the created external repository (default 'pip').
91-
**kwargs (dict): Keyword arguments passed directly to the `pip_repository` repository rule.
92+
**kwargs (dict): Additional arguments to the [`pip_repository`](./pip_repository.md) repository rule.
9293
"""
9394

9495
# Just in case our dependencies weren't already fetched
@@ -107,7 +108,8 @@ def pip_parse(requirements_lock, name = "pip_parsed_deps", **kwargs):
107108
Those dependencies become available in a generated `requirements.bzl` file.
108109
You can instead check this `requirements.bzl` file into your repo, see the "vendoring" section below.
109110
110-
This macro runs a repository rule that invokes `pip`. In your WORKSPACE file:
111+
This macro wraps the [`pip_repository`](./pip_repository.md) rule that invokes `pip`, with `incremental` set.
112+
In your WORKSPACE file:
111113
112114
```python
113115
load("@rules_python//python:pip.bzl", "pip_parse")
@@ -191,10 +193,11 @@ def pip_parse(requirements_lock, name = "pip_parsed_deps", **kwargs):
191193
of 'requirements' no resolve will take place and pip_repository will create
192194
individual repositories for each of your dependencies so that wheels are
193195
fetched/built only for the targets specified by 'build/run/test'.
196+
Note that if your lockfile is platform-dependent, you can use the `requirements_[platform]`
197+
attributes.
194198
name (str, optional): The name of the generated repository. The generated repositories
195199
containing each requirement will be of the form <name>_<requirement-name>.
196-
**kwargs (dict): Additional keyword arguments for the underlying
197-
`pip_repository` rule.
200+
**kwargs (dict): Additional arguments to the [`pip_repository`](./pip_repository.md) repository rule.
198201
"""
199202

200203
# Just in case our dependencies weren't already fetched

‎python/pip_install/pip_repository.bzl

Copy file name to clipboardExpand all lines: python/pip_install/pip_repository.bzl
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,7 @@ def _locked_requirements(rctx):
126126
requirements_txt = rctx.attr.requirements_darwin
127127
elif os.startswith("linux") and rctx.attr.requirements_linux != None:
128128
requirements_txt = rctx.attr.requirements_linux
129-
elif "win" in os:
129+
elif "win" in os and rctx.attr.requirements_windows != None:
130130
requirements_txt = rctx.attr.requirements_windows
131131
if not requirements_txt:
132132
fail("""\

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.