Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit bef2244

Browse filesBrowse files
UebelAndreUebelAndre
authored
pip_parse and pip_install can now parse entry points from wheels (bazel-contrib#523)
1 parent 9a10fdb commit bef2244
Copy full SHA for bef2244

File tree

13 files changed

+525
-29
lines changed
Filter options

13 files changed

+525
-29
lines changed

‎docs/pip.md

Copy file name to clipboard
100755100644
Expand all lines: docs/pip.md
+95-3Lines changed: 95 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,36 @@ py_library(
5151
)
5252
```
5353

54+
In addition to the `requirement` macro, which is used to access the generated `py_library`
55+
target generated from a package's wheel, The generated `requirements.bzl` file contains
56+
functionality for exposing [entry points][whl_ep] as `py_binary` targets as well.
57+
58+
[whl_ep]: https://packaging.python.org/specifications/entry-points/
59+
60+
```python
61+
load("@pip_deps//:requirements.bzl", "entry_point")
62+
63+
alias(
64+
name = "pip-compile",
65+
actual = entry_point(
66+
pkg = "pip-tools",
67+
script = "pip-compile",
68+
),
69+
)
70+
```
71+
72+
Note that for packages who's name and script are the same, only the name of the package
73+
is needed when calling the `entry_point` macro.
74+
75+
```python
76+
load("@pip_deps//:requirements.bzl", "entry_point")
77+
78+
alias(
79+
name = "flake8",
80+
actual = entry_point("flake8"),
81+
)
82+
```
83+
5484

5585
**PARAMETERS**
5686

@@ -70,16 +100,78 @@ py_library(
70100
pip_parse(<a href="#pip_parse-requirements_lock">requirements_lock</a>, <a href="#pip_parse-name">name</a>, <a href="#pip_parse-kwargs">kwargs</a>)
71101
</pre>
72102

103+
Imports a locked/compiled requirements file and generates a new `requirements.bzl` file.
104+
105+
This is used via the `WORKSPACE` pattern:
106+
107+
```python
108+
load("@rules_python//python:pip.bzl", "pip_parse")
109+
110+
pip_parse(
111+
name = "pip_deps",
112+
requirements_lock = ":requirements.txt",
113+
)
114+
115+
load("@pip_deps//:requirements.bzl", "install_deps")
116+
117+
install_deps()
118+
```
119+
120+
You can then reference imported dependencies from your `BUILD` file with:
121+
122+
```python
123+
load("@pip_deps//:requirements.bzl", "requirement")
124+
125+
py_library(
126+
name = "bar",
127+
...
128+
deps = [
129+
"//my/other:dep",
130+
requirement("requests"),
131+
requirement("numpy"),
132+
],
133+
)
134+
```
135+
136+
In addition to the `requirement` macro, which is used to access the generated `py_library`
137+
target generated from a package's wheel, The generated `requirements.bzl` file contains
138+
functionality for exposing [entry points][whl_ep] as `py_binary` targets as well.
139+
140+
[whl_ep]: https://packaging.python.org/specifications/entry-points/
141+
142+
```python
143+
load("@pip_deps//:requirements.bzl", "entry_point")
144+
145+
alias(
146+
name = "pip-compile",
147+
actual = entry_point(
148+
pkg = "pip-tools",
149+
script = "pip-compile",
150+
),
151+
)
152+
```
153+
154+
Note that for packages who's name and script are the same, only the name of the package
155+
is needed when calling the `entry_point` macro.
156+
157+
```python
158+
load("@pip_deps//:requirements.bzl", "entry_point")
159+
160+
alias(
161+
name = "flake8",
162+
actual = entry_point("flake8"),
163+
)
164+
```
73165

74166

75167
**PARAMETERS**
76168

77169

78170
| Name | Description | Default Value |
79171
| :-------------: | :-------------: | :-------------: |
80-
| requirements_lock | <p align="center"> - </p> | none |
81-
| name | <p align="center"> - </p> | <code>"pip_parsed_deps"</code> |
82-
| kwargs | <p align="center"> - </p> | none |
172+
| requirements_lock | A fully resolved 'requirements.txt' pip requirement file containing the transitive set of your dependencies. If this file is passed instead of 'requirements' no resolve will take place and pip_repository will create individual repositories for each of your dependencies so that wheels are fetched/built only for the targets specified by 'build/run/test'. | none |
173+
| name | The name of the generated repository. | <code>"pip_parsed_deps"</code> |
174+
| kwargs | Additional keyword arguments for the underlying <code>pip_repository</code> rule. | none |
83175

84176

85177
<a name="#pip_repositories"></a>

‎examples/pip_install/BUILD

Copy file name to clipboardExpand all lines: examples/pip_install/BUILD
+19-1Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
1-
load("@pip//:requirements.bzl", "requirement")
1+
load(
2+
"@pip//:requirements.bzl",
3+
"entry_point",
4+
"requirement",
5+
)
26
load("@rules_python//python:defs.bzl", "py_binary", "py_test")
37
load("@rules_python//python/pip_install:requirements.bzl", "compile_pip_requirements")
48

@@ -42,7 +46,21 @@ py_test(
4246
deps = [":main"],
4347
)
4448

49+
# For pip dependencies which have entry points, the `entry_point` macro can be
50+
# used from the generated `pip_install` repository to access a runnable binary.
51+
alias(
52+
name = "yamllint",
53+
actual = entry_point("yamllint"),
54+
)
55+
56+
py_test(
57+
name = "entry_point_test",
58+
srcs = ["entry_point_test.py"],
59+
data = [":yamllint"],
60+
)
61+
4562
# Check that our compiled requirements are up-to-date
4663
compile_pip_requirements(
4764
name = "requirements",
65+
extra_args = ["--allow-unsafe"],
4866
)

‎examples/pip_install/entry_point_test.py

Copy file name to clipboard
+20Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
#!/usr/bin/env python3
2+
3+
from pathlib import Path
4+
import subprocess
5+
import unittest
6+
7+
8+
class PipParseEntryPointTest(unittest.TestCase):
9+
def test_output(self):
10+
self.maxDiff = None
11+
12+
entry_point = Path("external/pip/pypi__yamllint/rules_python_wheel_entry_point_yamllint")
13+
self.assertTrue(entry_point.exists())
14+
15+
proc = subprocess.run([entry_point, "--version"], check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
16+
self.assertEqual(proc.stdout.decode("utf-8").strip(), "yamllint 1.26.3")
17+
18+
19+
if __name__ == "__main__":
20+
unittest.main()

‎examples/pip_install/requirements.in

Copy file name to clipboard
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
boto3==1.14.51
2+
yamllint==1.26.3

‎examples/pip_install/requirements.txt

Copy file name to clipboardExpand all lines: examples/pip_install/requirements.txt
+53-9Lines changed: 53 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -25,19 +25,63 @@ jmespath==0.10.0 \
2525
# via
2626
# boto3
2727
# botocore
28-
python-dateutil==2.8.1 \
29-
--hash=sha256:73ebfe9dbf22e832286dafa60473e4cd239f8592f699aa5adaf10050e6e1823c \
30-
--hash=sha256:75bb3f31ea686f1197762692a9ee6a7550b59fc6ca3a1f4b5d7e32fb98e2da2a
28+
pathspec==0.9.0 \
29+
--hash=sha256:7d15c4ddb0b5c802d161efc417ec1a2558ea2653c2e8ad9c19098201dc1c993a \
30+
--hash=sha256:e564499435a2673d586f6b2130bb5b95f04a3ba06f81b8f895b651a3c76aabb1
31+
# via yamllint
32+
python-dateutil==2.8.2 \
33+
--hash=sha256:0123cacc1627ae19ddf3c27a5de5bd67ee4586fbdd6440d9748f8abb483d3e86 \
34+
--hash=sha256:961d03dc3453ebbc59dbdea9e4e11c5651520a876d0f4db161e8674aae935da9
3135
# via botocore
32-
s3transfer==0.3.3 \
33-
--hash=sha256:2482b4259524933a022d59da830f51bd746db62f047d6eb213f2f8855dcb8a13 \
34-
--hash=sha256:921a37e2aefc64145e7b73d50c71bb4f26f46e4c9f414dc648c6245ff92cf7db
36+
pyyaml==5.4.1 \
37+
--hash=sha256:08682f6b72c722394747bddaf0aa62277e02557c0fd1c42cb853016a38f8dedf \
38+
--hash=sha256:0f5f5786c0e09baddcd8b4b45f20a7b5d61a7e7e99846e3c799b05c7c53fa696 \
39+
--hash=sha256:129def1b7c1bf22faffd67b8f3724645203b79d8f4cc81f674654d9902cb4393 \
40+
--hash=sha256:294db365efa064d00b8d1ef65d8ea2c3426ac366c0c4368d930bf1c5fb497f77 \
41+
--hash=sha256:3b2b1824fe7112845700f815ff6a489360226a5609b96ec2190a45e62a9fc922 \
42+
--hash=sha256:3bd0e463264cf257d1ffd2e40223b197271046d09dadf73a0fe82b9c1fc385a5 \
43+
--hash=sha256:4465124ef1b18d9ace298060f4eccc64b0850899ac4ac53294547536533800c8 \
44+
--hash=sha256:49d4cdd9065b9b6e206d0595fee27a96b5dd22618e7520c33204a4a3239d5b10 \
45+
--hash=sha256:4e0583d24c881e14342eaf4ec5fbc97f934b999a6828693a99157fde912540cc \
46+
--hash=sha256:5accb17103e43963b80e6f837831f38d314a0495500067cb25afab2e8d7a4018 \
47+
--hash=sha256:607774cbba28732bfa802b54baa7484215f530991055bb562efbed5b2f20a45e \
48+
--hash=sha256:6c78645d400265a062508ae399b60b8c167bf003db364ecb26dcab2bda048253 \
49+
--hash=sha256:72a01f726a9c7851ca9bfad6fd09ca4e090a023c00945ea05ba1638c09dc3347 \
50+
--hash=sha256:74c1485f7707cf707a7aef42ef6322b8f97921bd89be2ab6317fd782c2d53183 \
51+
--hash=sha256:895f61ef02e8fed38159bb70f7e100e00f471eae2bc838cd0f4ebb21e28f8541 \
52+
--hash=sha256:8c1be557ee92a20f184922c7b6424e8ab6691788e6d86137c5d93c1a6ec1b8fb \
53+
--hash=sha256:bb4191dfc9306777bc594117aee052446b3fa88737cd13b7188d0e7aa8162185 \
54+
--hash=sha256:bfb51918d4ff3d77c1c856a9699f8492c612cde32fd3bcd344af9be34999bfdc \
55+
--hash=sha256:c20cfa2d49991c8b4147af39859b167664f2ad4561704ee74c1de03318e898db \
56+
--hash=sha256:cb333c16912324fd5f769fff6bc5de372e9e7a202247b48870bc251ed40239aa \
57+
--hash=sha256:d2d9808ea7b4af864f35ea216be506ecec180628aced0704e34aca0b040ffe46 \
58+
--hash=sha256:d483ad4e639292c90170eb6f7783ad19490e7a8defb3e46f97dfe4bacae89122 \
59+
--hash=sha256:dd5de0646207f053eb0d6c74ae45ba98c3395a571a2891858e87df7c9b9bd51b \
60+
--hash=sha256:e1d4970ea66be07ae37a3c2e48b5ec63f7ba6804bdddfdbd3cfd954d25a82e63 \
61+
--hash=sha256:e4fac90784481d221a8e4b1162afa7c47ed953be40d31ab4629ae917510051df \
62+
--hash=sha256:fa5ae20527d8e831e8230cbffd9f8fe952815b2b7dae6ffec25318803a7528fc \
63+
--hash=sha256:fd7f6999a8070df521b6384004ef42833b9bd62cfee11a09bda1079b4b704247 \
64+
--hash=sha256:fdc842473cd33f45ff6bce46aea678a54e3d21f1b61a7750ce3c498eedfe25d6 \
65+
--hash=sha256:fe69978f3f768926cfa37b867e3843918e012cf83f680806599ddce33c2c68b0
66+
# via yamllint
67+
s3transfer==0.3.7 \
68+
--hash=sha256:35627b86af8ff97e7ac27975fe0a98a312814b46c6333d8a6b889627bcd80994 \
69+
--hash=sha256:efa5bd92a897b6a8d5c1383828dca3d52d0790e0756d49740563a3fb6ed03246
3570
# via boto3
36-
six==1.15.0 \
37-
--hash=sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259 \
38-
--hash=sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced
71+
six==1.16.0 \
72+
--hash=sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926 \
73+
--hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254
3974
# via python-dateutil
4075
urllib3==1.25.11 \
4176
--hash=sha256:8d7eaa5a82a1cac232164990f04874c594c9453ec55eef02eab885aa02fc17a2 \
4277
--hash=sha256:f5321fbe4bf3fefa0efd0bfe7fb14e90909eb62a48ccda331726b4319897dd5e
4378
# via botocore
79+
yamllint==1.26.3 \
80+
--hash=sha256:3934dcde484374596d6b52d8db412929a169f6d9e52e20f9ade5bf3523d9b96e
81+
# via -r requirements.in
82+
83+
# The following packages are considered to be unsafe in a requirements file:
84+
setuptools==57.5.0 \
85+
--hash=sha256:60d78588f15b048f86e35cdab73003d8b21dd45108ee61a6693881a427f22073 \
86+
--hash=sha256:d9d3266d50f59c6967b9312844470babbdb26304fe740833a5f8d89829ba3a24
87+
# via yamllint

‎examples/pip_parse/BUILD

Copy file name to clipboardExpand all lines: examples/pip_parse/BUILD
+28-1Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
1-
load("@pip_parsed_deps//:requirements.bzl", "requirement")
1+
load("@pip_parsed_deps//:requirements.bzl", "entry_point", "requirement")
22
load("@rules_python//python:defs.bzl", "py_binary", "py_test")
3+
load("@rules_python//python/pip_install:requirements.bzl", "compile_pip_requirements")
34

45
# Toolchain setup, this is optional.
56
# Demonstrate that we can use the same python interpreter for the toolchain and executing pip in pip install (see WORKSPACE).
@@ -40,3 +41,29 @@ py_test(
4041
srcs = ["test.py"],
4142
deps = [":main"],
4243
)
44+
45+
# For pip dependencies which have entry points, the `entry_point` macro can be
46+
# used from the generated `pip_parse` repository to access a runnable binary.
47+
alias(
48+
name = "yamllint",
49+
# If `pkg` and `script` are the same, passing a single string to
50+
# `entry_point` would work as well: `entry_point("yamllint")`
51+
actual = entry_point(
52+
pkg = "yamllint",
53+
script = "yamllint",
54+
),
55+
)
56+
57+
py_test(
58+
name = "entry_point_test",
59+
srcs = ["entry_point_test.py"],
60+
data = [":yamllint"],
61+
)
62+
63+
# This rule adds a convenient way to update the requiremenst file.
64+
compile_pip_requirements(
65+
name = "requirements",
66+
extra_args = ["--allow-unsafe"],
67+
requirements_in = "requirements.txt",
68+
requirements_txt = "requirements_lock.txt",
69+
)

‎examples/pip_parse/entry_point_test.py

Copy file name to clipboard
+20Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
#!/usr/bin/env python3
2+
3+
from pathlib import Path
4+
import subprocess
5+
import unittest
6+
7+
8+
class PipParseEntryPointTest(unittest.TestCase):
9+
def test_output(self):
10+
self.maxDiff = None
11+
12+
entry_point = Path("external/pip_parsed_deps_pypi__yamllint/rules_python_wheel_entry_point_yamllint")
13+
self.assertTrue(entry_point.exists())
14+
15+
proc = subprocess.run([entry_point, "--version"], check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
16+
self.assertEqual(proc.stdout.decode("utf-8").strip(), "yamllint 1.26.3")
17+
18+
19+
if __name__ == "__main__":
20+
unittest.main()

‎examples/pip_parse/requirements.txt

Copy file name to clipboard
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
requests==2.25.1
2+
yamllint==1.26.3

‎examples/pip_parse/requirements_lock.txt

Copy file name to clipboardExpand all lines: examples/pip_parse/requirements_lock.txt
+45-1Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
# This file is autogenerated by pip-compile
33
# To update, run:
44
#
5-
# pip-compile --generate-hashes --output-file=requirements_lock.txt requirements.txt
5+
# bazel run //:requirements.update
66
#
77
certifi==2020.12.5 \
88
--hash=sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c \
@@ -16,6 +16,41 @@ idna==2.10 \
1616
--hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6 \
1717
--hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0
1818
# via requests
19+
pathspec==0.9.0 \
20+
--hash=sha256:7d15c4ddb0b5c802d161efc417ec1a2558ea2653c2e8ad9c19098201dc1c993a \
21+
--hash=sha256:e564499435a2673d586f6b2130bb5b95f04a3ba06f81b8f895b651a3c76aabb1
22+
# via yamllint
23+
pyyaml==5.4.1 \
24+
--hash=sha256:08682f6b72c722394747bddaf0aa62277e02557c0fd1c42cb853016a38f8dedf \
25+
--hash=sha256:0f5f5786c0e09baddcd8b4b45f20a7b5d61a7e7e99846e3c799b05c7c53fa696 \
26+
--hash=sha256:129def1b7c1bf22faffd67b8f3724645203b79d8f4cc81f674654d9902cb4393 \
27+
--hash=sha256:294db365efa064d00b8d1ef65d8ea2c3426ac366c0c4368d930bf1c5fb497f77 \
28+
--hash=sha256:3b2b1824fe7112845700f815ff6a489360226a5609b96ec2190a45e62a9fc922 \
29+
--hash=sha256:3bd0e463264cf257d1ffd2e40223b197271046d09dadf73a0fe82b9c1fc385a5 \
30+
--hash=sha256:4465124ef1b18d9ace298060f4eccc64b0850899ac4ac53294547536533800c8 \
31+
--hash=sha256:49d4cdd9065b9b6e206d0595fee27a96b5dd22618e7520c33204a4a3239d5b10 \
32+
--hash=sha256:4e0583d24c881e14342eaf4ec5fbc97f934b999a6828693a99157fde912540cc \
33+
--hash=sha256:5accb17103e43963b80e6f837831f38d314a0495500067cb25afab2e8d7a4018 \
34+
--hash=sha256:607774cbba28732bfa802b54baa7484215f530991055bb562efbed5b2f20a45e \
35+
--hash=sha256:6c78645d400265a062508ae399b60b8c167bf003db364ecb26dcab2bda048253 \
36+
--hash=sha256:72a01f726a9c7851ca9bfad6fd09ca4e090a023c00945ea05ba1638c09dc3347 \
37+
--hash=sha256:74c1485f7707cf707a7aef42ef6322b8f97921bd89be2ab6317fd782c2d53183 \
38+
--hash=sha256:895f61ef02e8fed38159bb70f7e100e00f471eae2bc838cd0f4ebb21e28f8541 \
39+
--hash=sha256:8c1be557ee92a20f184922c7b6424e8ab6691788e6d86137c5d93c1a6ec1b8fb \
40+
--hash=sha256:bb4191dfc9306777bc594117aee052446b3fa88737cd13b7188d0e7aa8162185 \
41+
--hash=sha256:bfb51918d4ff3d77c1c856a9699f8492c612cde32fd3bcd344af9be34999bfdc \
42+
--hash=sha256:c20cfa2d49991c8b4147af39859b167664f2ad4561704ee74c1de03318e898db \
43+
--hash=sha256:cb333c16912324fd5f769fff6bc5de372e9e7a202247b48870bc251ed40239aa \
44+
--hash=sha256:d2d9808ea7b4af864f35ea216be506ecec180628aced0704e34aca0b040ffe46 \
45+
--hash=sha256:d483ad4e639292c90170eb6f7783ad19490e7a8defb3e46f97dfe4bacae89122 \
46+
--hash=sha256:dd5de0646207f053eb0d6c74ae45ba98c3395a571a2891858e87df7c9b9bd51b \
47+
--hash=sha256:e1d4970ea66be07ae37a3c2e48b5ec63f7ba6804bdddfdbd3cfd954d25a82e63 \
48+
--hash=sha256:e4fac90784481d221a8e4b1162afa7c47ed953be40d31ab4629ae917510051df \
49+
--hash=sha256:fa5ae20527d8e831e8230cbffd9f8fe952815b2b7dae6ffec25318803a7528fc \
50+
--hash=sha256:fd7f6999a8070df521b6384004ef42833b9bd62cfee11a09bda1079b4b704247 \
51+
--hash=sha256:fdc842473cd33f45ff6bce46aea678a54e3d21f1b61a7750ce3c498eedfe25d6 \
52+
--hash=sha256:fe69978f3f768926cfa37b867e3843918e012cf83f680806599ddce33c2c68b0
53+
# via yamllint
1954
requests==2.25.1 \
2055
--hash=sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804 \
2156
--hash=sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e
@@ -24,3 +59,12 @@ urllib3==1.26.5 \
2459
--hash=sha256:753a0374df26658f99d826cfe40394a686d05985786d946fbe4165b5148f5a7c \
2560
--hash=sha256:a7acd0977125325f516bda9735fa7142b909a8d01e8b2e4c8108d0984e6e0098
2661
# via requests
62+
yamllint==1.26.3 \
63+
--hash=sha256:3934dcde484374596d6b52d8db412929a169f6d9e52e20f9ade5bf3523d9b96e
64+
# via -r requirements.txt
65+
66+
# The following packages are considered to be unsafe in a requirements file:
67+
setuptools==57.5.0 \
68+
--hash=sha256:60d78588f15b048f86e35cdab73003d8b21dd45108ee61a6693881a427f22073 \
69+
--hash=sha256:d9d3266d50f59c6967b9312844470babbdb26304fe740833a5f8d89829ba3a24
70+
# via yamllint

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.