feat: implement basic pep440 support #1020

lukasbehammer · Sep 7, 2024

Purpose

Adds PEP440 support as dicussed in #1000, #1018 and #455

Rationale

I added version_compat as a cli flag and setting for branches. I also added PEP440 support, though not completely. Currently not added is support for multiple concurrent version specifiers (e.g. v1.2.3rc4.dev5) and support for epoch specifiers. Local versions have to be set manually via the buildmetadata option. Since the official regex was used, all possible pep440 version specifiers should at least be parsed from the tags. Setting the prerelease-token to one of the supported specifiers and setting as-prerelease makes a release with the token and prerelease version number 1. I have not yet implemented the auto-increase of the prerelease version. version_compat is not yet supported in the GitHub action. Docs are not yet updated.

How did you test?

I tested manually with a small test repository and command line. I have not yet written tests for the new functionality.

How to Verify

ToDo

Update Documentation
Add Github Action parameter
Add pep440 specific tests
Rename version_compat to a better name
Add support for multiple concurrent version specifiers
Add support for epoch specifiers
Add auto increase for prerelease version

lukasbehammer · Sep 7, 2024

This is not yet ready to be merged. I'll have to write some tests first and some of the tests fail currently. @codejedi365 tell me what you think about this implementation and I'll go ahead writing tests, improving it and adding it to the Actions.

lukasbehammer · Sep 25, 2024

Just a quick reminder @codejedi365 if you can have a quick look at this implementation of pep440 support, so that I know if I should go forward with the implementation.

codejedi365

I agree with your approach. I'm glad you broke out a separate regex just for pep440 and I liked how you used the version translator as well.

I do think where you have the type string for all the version_compat vars should be a defined enum as we strictly only have the 2. Also is there a better name we can come up with?

My other thought is I'm not sure how much version_compat parameters we need throughout the code. I would rather attempt to read semver then attempt to read pep404 when parsing the tag history (tags_and_versions()) and then convert that into a Version object with a flag set to if the tag was a pep440 or not. That way we can actually support all tags (even in the odd chance that the user switched from one to the other). Then like, you already did, adjust the Version.__str__ method to print out a pep440 variant or a semver version depending on what configuration the user has set for what version types they want to use for the new versions. I would like to see it as a --pep440 option to the version command, which would print any Version object to pep440 (as long as its compatible tokens).

Otherwise well done and make sure to add tests, update the documentation and add a GitHub action parameter. If you need help with this, let me know. Also just as a heads up I have a few PRs I'm working simultaneously so there may be some conflicts as this PR likely will be quite large. If it makes more sense, we should probably break it up into smaller doses and merge parts (given the functionality is not accessible to the user, ie. last thing to actually add is to enable the --pep440 option).

codejedi365 · Sep 25, 2024

semantic_release/version/translator.py

@@ -44,10 +41,12 @@ def __init__(
        self,
        tag_format: str = "v{version}",
        prerelease_token: str = "rc",  # noqa: S107
+        version_compat: str = "semver"


probably should be a enum as we have strictly 2 possibilities

codejedi365 · Sep 25, 2024

Sorry for the delay, I was sick and then out of town for a week. Thanks for the reminder.

codejedi365 · Oct 15, 2024

@lukasbehammer, did you have questions on my feedback?

lukasbehammer · Oct 15, 2024

Ah sorry @codejedi365 . I haven't had much time lately. The earliest time I can do some more is probably around December/January. Though I'll try to fit some minor stuff, like tests etc. in such that it's easier to merge. If you have some time feel free to continue with it.

lukasbehammer · Oct 19, 2024

I changed the existing tests to run with the new version-compat scheme, though I'm not sure how to implement the tests for the pep440 compatibility. Should we copy the semver ones and run them again with pep440? What is the best approach?

codejedi365 · Oct 19, 2024

Should we copy the semver ones and run them again with pep440? What is the best approach?

For unit testing:

you will want to have a variant of the semver parse testing to ensure pep440 strings are parsed properly
You should probably add some comparator testing between a semver and pep440 parsed version
Likely need to add a set of bump tests to ensure the resulting string is pep440 compliant

For system testing:

Add to the tests/command_line/ directory

Test the --pep440 option that it generates a pep440 version the following repos variants

all pep440 versions (this fixture doesn't exist, duplicate the repo with angular commits type, maybe a few of them since prereleases versions must be tested)
mixed pep440 and semver versions where someone started with pep440 and converted to semver
mixed pep440 and semver versions where someone started with semver and converted to pep440

Add the repo variants to the changelog_regenerate test
Add the new pep440 repos to any changelog update tests that make sense
Make sure publish command works when pointing at a pep440 version
Make sure changelog post to release tag works when it's a pep440 version

This is what I came up with so far from memory, if I think of something else I'll post it. It's a lot because this is like a core functionality and interoperability change. I appreciate the hard work and time you have put in.

codejedi365 · Oct 19, 2024

Remember to rebase soon as there have been some changes your branch conflicts with

lukasbehammer · Oct 19, 2024

@codejedi365 regarding the --pep440 flag you suggested, I'd rather keep the --version-compat flag such that we're open to future versioning schemes.

lukasbehammer · Oct 19, 2024

Remember to rebase soon as there have been some changes your branch conflicts with

I'll do.

codejedi365 · Oct 19, 2024

@codejedi365 regarding the --pep440 flag you suggested, I'd rather keep the --version-compat flag such that we're open to future versioning schemes.

Although I appreciate the forward look, I think the current name is less intuitive and I don't anticipate any more version schemes as the world has settled around semver fairly heavily. I know there is CalVer out there but that isn't directly compatible with our version bump construct. Any other version scheme would need to match the version bump construct. With that said, I stand by my comment to change to --pep440 as the option. If my predictions is wrong, I'll deal with it when the time comes.

codejedi365 · Oct 19, 2024

Added another unit test idea to the list above.

lukasbehammer · Oct 19, 2024

Is version_scheme better than version_compat? If so I'd refactor.

codejedi365 · Oct 19, 2024

@lukasbehammer, when you start running the tests you are probably going to want to add the -nauto flag for the xdist plugin to use multiple cpus. You probably will want to isolate the specific test files to run or test case while running locally because the whole suite takes a long time.

It's on the list of things to fix but it's not quite easy to change.

lukasbehammer · Oct 19, 2024

Also I tried to implement the Enum. Didn't really work out. Maybe you @codejedi365 can have a look at it.

codejedi365 · Oct 20, 2024

Sure, I can take a look tomorrow

cfxegbert · Jan 7, 2025

Should version_compat be a config file option instead of both a command line option and a config file option? It would seem error prone to have to specify on the command line every time you bump the version. You would just want to configure it once in your pyproject.toml or releaserc.toml file. In my opinion version_compat should only be a config file option. It should not be both a cli and config option.

cfxegbert · Jan 7, 2025

src/semantic_release/version/version.py

-        )
+        if self.version_compat == "semver":
+            pre_str = f"-{self.prerelease_token}.{self.prerelease_revision}"
+        elif self.version_compat == "pep440":


Since you are tranforming the prerelease_token in a lossy way the --print_tag command line option may not actually represent the actual tag that will be stored in the repo. The print_tag takes the str(version) and converts it to a tag. I would not do the transformation and keep 'alpha', 'beta', ... since they are allowed by pep440.

We should implement unit test for this case.

If this being done for normalization for sorting we should make a normalize method on version just for sorting.

Since you are tranforming the prerelease_token in a lossy way the --print_tag command line option may not actually represent the actual tag that will be stored in the repo. The print_tag takes the str(version) and converts it to a tag.

This is a good catch but actually unrelated to this PR. This is a bug and will be fixed separately (#1134). It is supposed to provide the actual tag that would match the repo. An e2e test will be written for this bug fix.

Yes the planned implementation is to evaluate the repo for all release tags and convert them to Version Objects. First we try the SEMVER parser and then secondly we try a pep440 parser to normalize all versions that match a tag format. This way sorting is correct regardless of what version spec they used.

The resulting tag or printed version (and/or tag) is where the --pep440 parameter and a configuration option tweak PSR to output a pep440 formatted version.

This is a good catch but actually unrelated to this PR. This is a bug and will be fixed separately

That fix still converts the version to a string so this transformation when converting to a string will still break producing the next tag. The only reason to normalize the version is for sorting.

cfxegbert · Jan 7, 2025

Current implementation has a hardcoded separator between the the release and pre/post/dev token and the separator between pre/post/dev and the number. In pep440 "1.2.3-pre-1" is the same as "1.2.3pre.1". Should we have a config option for the separators? We could use the already existing prerelease option and allow something like

[semantic_release.branches.dev]
match = "develop"
prerelease_token = "-dev."
prerelease = true

The above would generate 1.2.3-dev.1 and a prerelease_token = "_dev" would generate 1.2.3_dev1. The checks on allowed prerelease_tokens could be done after the config file is parsed. This would also easily handle the post release format of 1.2.3-1

We would just have to document it correctly.

codejedi365 · Jan 8, 2025

Should version_compat be a config file option instead of both a command line option and a config file option? It would seem error prone to have to specify on the command line every time you bump the version. You would just want to configure it once in your pyproject.toml or releaserc.toml file. In my opinion version_compat should only be a config file option. It should not be both a cli and config option.

Yes, we should probably have a configuration option in the config for it as it likely is a lasting setting. I still would like to maintain the cli option as that is how we provide overrides to the github action. The github action is almost configureless as the defaults will load accordingly. We will not make pep440 the default so it will need to be a cli option.

codejedi365 · Jan 8, 2025

Current implementation has a hardcoded separator between the the release and pre/post/dev token and the separator between pre/post/dev and the number. In pep440 "1.2.3-pre-1" is the same as "1.2.3pre.1". Should we have a config option for the separators? We could use the already existing prerelease option and allow something like
[semantic_release.branches.dev]
match = "develop"
prerelease_token = "-dev."
prerelease = true
The above would generate 1.2.3-dev.1 and a prerelease_token = "_dev" would generate 1.2.3_dev1. The checks on allowed prerelease_tokens could be done after the config file is parsed. This would also easily handle the post release format of 1.2.3-1

We would just have to document it correctly.

I do not intend to provide configuration for the separator as that would result in non-conformative versions to the two supported standards. The reason it is hardcoded is because PSR builds SemVer compliant versions and once this PR is complete, it will have the ability to create PEP440 compliant versions. I will have to check on those version similarities. I hope not... I just can't stand pep440.

cfxegbert · Jan 10, 2025

Should version_compat be a config file option instead of both a command line option and a config file option? It would seem error prone to have to specify on the command line every time you bump the version. You would just want to configure it once in your pyproject.toml or releaserc.toml file. In my opinion version_compat should only be a config file option. It should not be both a cli and config option.

Yes, we should probably have a configuration option in the config for it as it likely is a lasting setting. I still would like to maintain the cli option as that is how we provide overrides to the github action. The github action is almost configureless as the defaults will load accordingly. We will not make pep440 the default so it will need to be a cli option.

Having a command line flag would be inconsistent with every other setting. Why could the config file settings not work in a github action?

lukasbehammer · Jan 10, 2025

Having a command line flag would be inconsistent with every other setting. Why could the config file settings not work in a github action?

I don't think the command line flag is inconsistent. All the other things can be either set by a config file (e.g. pyproject.toml) or by command line options. The GitHub action currently indeed is setup to use the cli options to call the code.

On another note: I still prefer the version_compat scheme, but have no obligations against using a pep440 flag and trying to evaluate the semver first.

codejedi365 · Jan 11, 2025

Having a command line flag would be inconsistent with every other setting.

That is a statement of opinion that I do not share. I consider each option and its usage independently for when it should be available as a cli flag.

Why could the config file settings not work in a github action?

I did not say that. Configuration files are parsed in a GitHub Action. Cli options are for overrides. Just like in every other Linux-flavored-cli tool. And I said the plan is to provide both.

codejedi365 · Jan 11, 2025

Also I tried to implement the Enum. Didn't really work out. Maybe you @codejedi365 can have a look at it.

@lukasbehammer, sorry I didn't get back around to this, I've been working other fixes & features that have been lingering too long. Hopefully over the next month.

add version-compat flag to commandline and config to choose a versioning scheme, implement basic pep440 support

github-actions · Apr 7, 2025

It has been 60 days since the last update on this confirmed PR. @python-semantic-release/team can you provide an update on the status of this PR?

codejedi365 self-requested a review September 25, 2024 17:29

codejedi365 reviewed Sep 25, 2024

View reviewed changes

lukasbehammer marked this pull request as draft October 19, 2024 14:57

lukasbehammer force-pushed the dev-pep440 branch from d987b8f to 875a5a2 Compare October 19, 2024 21:02

codejedi365 mentioned this pull request Oct 30, 2024

feat: stamp a canonical dev version instead of semver #1000

Open

codejedi365 force-pushed the dev-pep440 branch 2 times, most recently from b03d552 to 2cff31d Compare November 10, 2024 08:03

codejedi365 force-pushed the dev-pep440 branch from 2cff31d to 5a3d9ab Compare December 3, 2024 08:13

codejedi365 added the confirmed Prevent from becoming stale label Dec 16, 2024

cfxegbert reviewed Jan 7, 2025

View reviewed changes

codejedi365 force-pushed the dev-pep440 branch from 5a3d9ab to b541bd7 Compare January 12, 2025 22:36

codejedi365 force-pushed the dev-pep440 branch from b541bd7 to 1fb70b7 Compare January 26, 2025 02:30

lukasbehammer added 4 commits February 6, 2025 00:14

test: update tests with new "version-compat" scheme

aa7ac1c

feat: add basic pep440 compatibility

acea362

add version-compat flag to commandline and config to choose a versioning scheme, implement basic pep440 support

feat(github-actions): add pep440 parameter to GitHub Action inputs

7535d3b

chore(config): update ruff command in tox config

ae71d43

codejedi365 force-pushed the dev-pep440 branch from 1fb70b7 to ae71d43 Compare February 6, 2025 05:15

github-actions bot added the needs-update Needs status update from maintainers label Apr 7, 2025

Search code, repositories, users, issues, pull requests...

feat: implement basic pep440 support #1020

Are you sure you want to change the base?

feat: implement basic pep440 support #1020

Uh oh!

Conversation

lukasbehammer commented Sep 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Rationale

How did you test?

How to Verify

ToDo

Uh oh!

lukasbehammer commented Sep 7, 2024

Uh oh!

lukasbehammer commented Sep 25, 2024

Uh oh!

codejedi365 left a comment

Choose a reason for hiding this comment

Uh oh!

codejedi365 Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

codejedi365 commented Sep 25, 2024

Uh oh!

codejedi365 commented Oct 15, 2024

Uh oh!

lukasbehammer commented Oct 15, 2024

Uh oh!

lukasbehammer commented Oct 19, 2024

Uh oh!

codejedi365 commented Oct 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codejedi365 commented Oct 19, 2024

Uh oh!

lukasbehammer commented Oct 19, 2024

Uh oh!

lukasbehammer commented Oct 19, 2024

Uh oh!

codejedi365 commented Oct 19, 2024

Uh oh!

codejedi365 commented Oct 19, 2024

Uh oh!

lukasbehammer commented Oct 19, 2024

Uh oh!

codejedi365 commented Oct 19, 2024

Uh oh!

lukasbehammer commented Oct 19, 2024

Uh oh!

codejedi365 commented Oct 20, 2024

Uh oh!

cfxegbert commented Jan 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cfxegbert Jan 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cfxegbert Jan 7, 2025

Choose a reason for hiding this comment

Uh oh!

codejedi365 Jan 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codejedi365 Jan 8, 2025

Choose a reason for hiding this comment

Uh oh!

cfxegbert Jan 11, 2025

Choose a reason for hiding this comment

Uh oh!

cfxegbert commented Jan 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codejedi365 commented Jan 8, 2025

Uh oh!

codejedi365 commented Jan 8, 2025

Uh oh!

lukasbehammer commented Sep 7, 2024 •

edited

Loading

codejedi365 commented Oct 19, 2024 •

edited

Loading

cfxegbert commented Jan 7, 2025 •

edited

Loading

cfxegbert Jan 7, 2025 •

edited

Loading

codejedi365 Jan 8, 2025 •

edited

Loading

cfxegbert commented Jan 7, 2025 •

edited

Loading

codejedi365 commented Jan 11, 2025 •

edited

Loading