Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit dae610b

Browse filesBrowse files
mattoberleJonathon Belottialexeagle
authored
Use PEP 508 rules when setting deps from extras (bazel-contrib#724)
* Use PEP 426 rules when setting deps from extras This commit addresses issue bazel-contrib#720. [PEP 426](https://peps.python.org/pep-0426/#name) states that distribution names are case-insensitive and "-" is interchangeable with "_". The `pip-compile` command creates a lockfile where all package names are lowercase. The tool may also modify interchangeable characters. The following examples are all valid `requirements.txt` or `requirements_lock.txt` entries: ``` SQLAlchemy[postgresql_psycopg2binary]==1.4.36 sqlalchemy[postgresql_psycopg2binary]==1.4.36 sentry_sdk[flask]==1.5.8 sentry-sdk[flask]==1.5.8 ``` A distribution's `METADATA` file contains the stylization chosen by the publisher. By applying a "sanitise" function when building the `extras` dict and when performing lookups we can eliminate this difference as a concern. * Use PEP 503 rules when sanitising extras * Normalize distribution name with pkg_resources `pypa/installer` is used to parse Wheel metadata, but does not currently provide a method for normalizing distribution names: - pypa/installer#97 `pypa/pkg_resources` provides `Requirement.parse` which returns an instance of `Requirement` where `.key` is the canonical distribution name per PEP 503. The `Requirement` class can also parse `extras`, but it returns a normalized form that I believe could break the installation of the extras. * Use Requirement.parse to populate extra reqs * Revert "Use Requirement.parse to populate extra reqs" This reverts commit f0faa97. * Test for distribution name normalization in extras * Replace pkg_resources with packaging.utils This replaces `pkg_resources.Requirement.parse` with `packaging.utils.canonicalize_name`. Doing this pulls in a vendored requirement from `pip`, which may be undesirable. The code we want is just: ``` re.sub(r"[-_.]+", "-", name).lower() ``` This commit also leaves a reference to `pkg_resources` in `wheel.py` which does not canonicalize the name. Co-authored-by: Jonathon Belotti <jonathon@canva.com> Co-authored-by: Alex Eagle <alex@aspect.dev>
1 parent c98bc8f commit dae610b
Copy full SHA for dae610b

File tree

3 files changed

+8
-2
lines changed
Filter options

3 files changed

+8
-2
lines changed

‎python/pip_install/extract_wheels/lib/requirements.py

Copy file name to clipboardExpand all lines: python/pip_install/extract_wheels/lib/requirements.py
+3-1Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
import re
22
from typing import Dict, Optional, Set, Tuple
33

4+
from pip._vendor.packaging.utils import canonicalize_name
5+
46

57
def parse_extras(requirements_path: str) -> Dict[str, Set[str]]:
68
"""Parse over the requirements.txt file to find extras requested.
@@ -38,7 +40,7 @@ def _parse_requirement_for_extra(
3840
matches = extras_pattern.match(requirement)
3941
if matches:
4042
return (
41-
matches.group(1),
43+
canonicalize_name(matches.group(1)),
4244
{extra.strip() for extra in matches.group(2).split(",")},
4345
)
4446

‎python/pip_install/extract_wheels/lib/requirements_test.py

Copy file name to clipboardExpand all lines: python/pip_install/extract_wheels/lib/requirements_test.py
+2Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,8 @@ def test_parses_requirement_for_extra(self) -> None:
99
("name[foo]", ("name", frozenset(["foo"]))),
1010
("name[ Foo123 ]", ("name", frozenset(["Foo123"]))),
1111
(" name1[ foo ] ", ("name1", frozenset(["foo"]))),
12+
("Name[foo]", ("name", frozenset(["foo"]))),
13+
("name_foo[bar]", ("name-foo", frozenset(["bar"]))),
1214
(
1315
"name [fred,bar] @ http://foo.com ; python_version=='2.7'",
1416
("name", frozenset(["fred", "bar"])),

‎python/pip_install/extract_wheels/lib/wheel.py

Copy file name to clipboardExpand all lines: python/pip_install/extract_wheels/lib/wheel.py
+3-1Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88

99
import installer
1010
import pkg_resources
11+
from pip._vendor.packaging.utils import canonicalize_name
1112

1213

1314
def current_umask() -> int:
@@ -38,7 +39,8 @@ def path(self) -> str:
3839
@property
3940
def name(self) -> str:
4041
# TODO Also available as installer.sources.WheelSource.distribution
41-
return str(self.metadata['Name'])
42+
name = str(self.metadata['Name'])
43+
return canonicalize_name(name)
4244

4345
@property
4446
def metadata(self) -> email.message.Message:

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.