Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit 20ac9bc

Browse filesBrowse files
rickeylevaignas
andauthored
feat(rules): allow deriving custom rules from core rules (#2666)
This exposes public functions for creating builders for py_binary, py_test, and py_library. It also adds some docs and examples for how to use them. I'm calling this a "volatile" API -- it's public, but the pieces that comprise it (e.g. all the rule args, attributes, the attribute args, etc) are likely to change in various ways, and not all modifications to them can be supported in a backward compatible way. Hence the "volatile" term: * hold it gently and its fine * shake it a bit and its probably fine * shake it moderately and something may or may not blow up * shake it a lot and something will certainly blow up. Work towards #1647 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>
1 parent c0b5075 commit 20ac9bc
Copy full SHA for 20ac9bc
Expand file treeCollapse file tree

17 files changed

+321
-54
lines changed

‎CHANGELOG.md

Copy file name to clipboardExpand all lines: CHANGELOG.md
+3Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,9 @@ Unreleased changes template.
9191
* (pypi) Direct HTTP urls for wheels and sdists are now supported when using
9292
{obj}`experimental_index_url` (bazel downloader).
9393
Partially fixes [#2363](https://github.com/bazelbuild/rules_python/issues/2363).
94+
* (rules) APIs for creating custom rules based on the core py_binary, py_test,
95+
and py_library rules
96+
([#1647](https://github.com/bazelbuild/rules_python/issues/1647))
9497

9598
{#v0-0-0-removed}
9699
### Removed

‎docs/BUILD.bazel

Copy file name to clipboardExpand all lines: docs/BUILD.bazel
+2Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,8 @@ sphinx_stardocs(
100100
"//python:py_test_bzl",
101101
"//python:repositories_bzl",
102102
"//python/api:api_bzl",
103+
"//python/api:executables_bzl",
104+
"//python/api:libraries_bzl",
103105
"//python/cc:py_cc_toolchain_bzl",
104106
"//python/cc:py_cc_toolchain_info_bzl",
105107
"//python/entry_points:py_console_script_binary_bzl",

‎docs/_includes/volatile_api.md

Copy file name to clipboard
+5Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
:::{important}
2+
3+
**Public, but volatile, API.** Some parts are stable, while others are
4+
implementation details and may change more frequently.
5+
:::

‎docs/extending.md

Copy file name to clipboard
+143Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
# Extending the rules
2+
3+
:::{important}
4+
**This is public, but volatile, functionality.**
5+
6+
Extending and customizing the rules is supported functionality, but with weaker
7+
backwards compatibility guarantees, and is not fully subject to the normal
8+
backwards compatibility procedures and policies. It's simply not feasible to
9+
support every possible customization with strong backwards compatibility
10+
guarantees.
11+
:::
12+
13+
Because of the rich ecosystem of tools and variety of use cases, APIs are
14+
provided to make it easy to create custom rules using the existing rules as a
15+
basis. This allows implementing behaviors that aren't possible using
16+
wrapper macros around the core rules, and can make certain types of changes
17+
much easier and transparent to implement.
18+
19+
:::{note}
20+
It is not required to extend a core rule. The minimum requirement for a custom
21+
rule is to return the appropriate provider (e.g. {bzl:obj}`PyInfo` etc).
22+
Extending the core rules is most useful when you want all or most of the
23+
behavior of a core rule.
24+
:::
25+
26+
Follow or comment on https://github.com/bazelbuild/rules_python/issues/1647
27+
for the development of APIs to support custom derived rules.
28+
29+
## Creating custom rules
30+
31+
Custom rules can be created using the core rules as a basis by using their rule
32+
builder APIs.
33+
34+
* [`//python/apis:executables.bzl`](#python-apis-executables-bzl): builders for
35+
executables.
36+
* [`//python/apis:libraries.bzl`](#python-apis-libraries-bzl): builders for
37+
libraries.
38+
39+
These builders create {bzl:obj}`ruleb.Rule` objects, which are thin
40+
wrappers around the keyword arguments eventually passed to the `rule()`
41+
function. These builder APIs give access to the _entire_ rule definition and
42+
allow arbitrary modifications.
43+
44+
This is level of control is powerful, but also volatile. A rule definition
45+
contains many details that _must_ change as the implementation changes. What
46+
is more or less likely to change isn't known in advance, but some general
47+
rules are:
48+
49+
* Additive behavior to public attributes will be less prone to breaking.
50+
* Internal attributes that directly support a public attribute are likely
51+
reliable.
52+
* Internal attributes that support an action are more likely to change.
53+
* Rule toolchains are moderately stable (toolchains are mostly internal to
54+
how a rule works, but custom toolchains are supported).
55+
56+
## Example: validating a source file
57+
58+
In this example, we derive from `py_library` a custom rule that verifies source
59+
code contains the word "snakes". It does this by:
60+
61+
* Adding an implicit dependency on a checker program
62+
* Calling the base implementation function
63+
* Running the checker on the srcs files
64+
* Adding the result to the `_validation` output group (a special output
65+
group for validation behaviors).
66+
67+
To users, they can use `has_snakes_library` the same as `py_library`. The same
68+
is true for other targets that might consume the rule.
69+
70+
```
71+
load("@rules_python//python/api:libraries.bzl", "libraries")
72+
load("@rules_python//python/api:attr_builders.bzl", "attrb")
73+
74+
def _has_snakes_impl(ctx, base):
75+
providers = base(ctx)
76+
77+
out = ctx.actions.declare_file(ctx.label.name + "_snakes.check")
78+
ctx.actions.run(
79+
inputs = ctx.files.srcs,
80+
outputs = [out],
81+
executable = ctx.attr._checker[DefaultInfo].files_to_run,
82+
args = [out.path] + [f.path for f in ctx.files.srcs],
83+
)
84+
prior_ogi = None
85+
for i, p in enumerate(providers):
86+
if type(p) == "OutputGroupInfo":
87+
prior_ogi = (i, p)
88+
break
89+
if prior_ogi:
90+
groups = {k: getattr(prior_ogi[1], k) for k in dir(prior_ogi)}
91+
if "_validation" in groups:
92+
groups["_validation"] = depset([out], transitive=groups["_validation"])
93+
else:
94+
groups["_validation"] = depset([out])
95+
providers[prior_ogi[0]] = OutputGroupInfo(**groups)
96+
else:
97+
providers.append(OutputGroupInfo(_validation=depset([out])))
98+
return providers
99+
100+
def create_has_snakes_rule():
101+
r = libraries.py_library_builder()
102+
base_impl = r.implementation()
103+
r.set_implementation(lambda ctx: _has_snakes_impl(ctx, base_impl))
104+
r.attrs["_checker"] = attrb.Label(
105+
default="//:checker",
106+
executable = True,
107+
)
108+
return r.build()
109+
has_snakes_library = create_has_snakes_rule()
110+
```
111+
112+
## Example: adding transitions
113+
114+
In this example, we derive from `py_binary` to force building for a particular
115+
platform. We do this by:
116+
117+
* Adding an additional output to the rule's cfg
118+
* Calling the base transition function
119+
* Returning the new transition outputs
120+
121+
```starlark
122+
123+
load("@rules_python//python/api:executables.bzl", "executables")
124+
125+
def _force_linux_impl(settings, attr, base_impl):
126+
settings = base_impl(settings, attr)
127+
settings["//command_line_option:platforms"] = ["//my/platforms:linux"]
128+
return settings
129+
130+
def create_rule():
131+
r = executables.py_binary_rule_builder()
132+
base_impl = r.cfg.implementation()
133+
r.cfg.set_implementation(
134+
lambda settings, attr: _force_linux_impl(settings, attr, base_impl)
135+
)
136+
r.cfg.add_output("//command_line_option:platforms")
137+
return r.build()
138+
139+
py_linux_binary = create_linux_binary_rule()
140+
```
141+
142+
Users can then use `py_linux_binary` the same as a regular py_binary. It will
143+
act as if `--platforms=//my/platforms:linux` was specified when building it.

‎docs/index.md

Copy file name to clipboardExpand all lines: docs/index.md
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,7 @@ pip
101101
coverage
102102
precompiling
103103
gazelle
104+
Extending <extending>
104105
Contributing <contributing>
105106
support
106107
Changelog <changelog>

‎python/api/BUILD.bazel

Copy file name to clipboardExpand all lines: python/api/BUILD.bazel
+20Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,26 @@ bzl_library(
2525
deps = ["//python/private/api:api_bzl"],
2626
)
2727

28+
bzl_library(
29+
name = "executables_bzl",
30+
srcs = ["executables.bzl"],
31+
visibility = ["//visibility:public"],
32+
deps = [
33+
"//python/private:py_binary_rule_bzl",
34+
"//python/private:py_executable_bzl",
35+
"//python/private:py_test_rule_bzl",
36+
],
37+
)
38+
39+
bzl_library(
40+
name = "libraries_bzl",
41+
srcs = ["libraries.bzl"],
42+
visibility = ["//visibility:public"],
43+
deps = [
44+
"//python/private:py_library_bzl",
45+
],
46+
)
47+
2848
filegroup(
2949
name = "distribution",
3050
srcs = glob(["**"]),

‎python/api/executables.bzl

Copy file name to clipboard
+31Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Copyright 2025 The Bazel Authors. All rights reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""
16+
{#python-apis-executables-bzl}
17+
Loading-phase APIs specific to executables (binaries/tests).
18+
19+
:::{versionadded} VERSION_NEXT_FEATURE
20+
:::
21+
"""
22+
23+
load("//python/private:py_binary_rule.bzl", "create_py_binary_rule_builder")
24+
load("//python/private:py_executable.bzl", "create_executable_rule_builder")
25+
load("//python/private:py_test_rule.bzl", "create_py_test_rule_builder")
26+
27+
executables = struct(
28+
py_binary_rule_builder = create_py_binary_rule_builder,
29+
py_test_rule_builder = create_py_test_rule_builder,
30+
executable_rule_builder = create_executable_rule_builder,
31+
)

‎python/api/libraries.bzl

Copy file name to clipboard
+27Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Copyright 2025 The Bazel Authors. All rights reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""
16+
{#python-apis-libraries-bzl}
17+
Loading-phase APIs specific to libraries.
18+
19+
:::{versionadded} VERSION_NEXT_FEATURE
20+
:::
21+
"""
22+
23+
load("//python/private:py_library.bzl", "create_py_library_rule_builder")
24+
25+
libraries = struct(
26+
py_library_rule_builder = create_py_library_rule_builder,
27+
)

‎python/private/BUILD.bazel

Copy file name to clipboardExpand all lines: python/private/BUILD.bazel
+1-2Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -427,6 +427,7 @@ bzl_library(
427427
":attributes_bzl",
428428
":common_bzl",
429429
":flags_bzl",
430+
":precompile_bzl",
430431
":py_cc_link_params_info_bzl",
431432
":py_internal_bzl",
432433
":rule_builders_bzl",
@@ -446,8 +447,6 @@ bzl_library(
446447
name = "py_library_rule_bzl",
447448
srcs = ["py_library_rule.bzl"],
448449
deps = [
449-
":common_bzl",
450-
":precompile_bzl",
451450
":py_library_bzl",
452451
],
453452
)

‎python/private/attr_builders.bzl

Copy file name to clipboardExpand all lines: python/private/attr_builders.bzl
+5-1Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,11 @@
1212
# See the License for the specific language governing permissions and
1313
# limitations under the License.
1414

15-
"""Builders for creating attributes et al."""
15+
"""Builders for creating attributes et al.
16+
17+
:::{versionadded} VERSION_NEXT_FEATURE
18+
:::
19+
"""
1620

1721
load("@bazel_skylib//lib:types.bzl", "types")
1822
load(

‎python/private/py_binary_rule.bzl

Copy file name to clipboardExpand all lines: python/private/py_binary_rule.bzl
+15-2Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,12 +27,25 @@ def _py_binary_impl(ctx):
2727
inherited_environment = [],
2828
)
2929

30-
def create_binary_rule_builder():
30+
# NOTE: Exported publicly
31+
def create_py_binary_rule_builder():
32+
"""Create a rule builder for a py_binary.
33+
34+
:::{include} /_includes/volatile_api.md
35+
:::
36+
37+
:::{versionadded} VERSION_NEXT_FEATURE
38+
:::
39+
40+
Returns:
41+
{type}`ruleb.Rule` with the necessary settings
42+
for creating a `py_binary` rule.
43+
"""
3144
builder = create_executable_rule_builder(
3245
implementation = _py_binary_impl,
3346
executable = True,
3447
)
3548
builder.attrs.update(AGNOSTIC_BINARY_ATTRS)
3649
return builder
3750

38-
py_binary = create_binary_rule_builder().build()
51+
py_binary = create_py_binary_rule_builder().build()

‎python/private/py_executable.bzl

Copy file name to clipboardExpand all lines: python/private/py_executable.bzl
+17Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1737,7 +1737,24 @@ def create_base_executable_rule():
17371737
"""
17381738
return create_executable_rule_builder().build()
17391739

1740+
# NOTE: Exported publicly
17401741
def create_executable_rule_builder(implementation, **kwargs):
1742+
"""Create a rule builder for an executable Python program.
1743+
1744+
:::{include} /_includes/volatile_api.md
1745+
:::
1746+
1747+
An executable rule is one that sets either `executable=True` or `test=True`,
1748+
and the output is something that can be run directly (e.g. `bazel run`,
1749+
`exec(...)` etc)
1750+
1751+
:::{versionadded} VERSION_NEXT_FEATURE
1752+
:::
1753+
1754+
Returns:
1755+
{type}`ruleb.Rule` with the necessary settings
1756+
for creating an executable Python rule.
1757+
"""
17411758
builder = ruleb.Rule(
17421759
implementation = implementation,
17431760
attrs = EXECUTABLE_ATTRS,

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.