Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Question about using a front end like PDM with meson-python #273

Answered by eli-schwartz
lwasser asked this question in Q&A
Discussion options

Hi there! @eli-schwartz i'm following up on our convos in the open PR i'm working on surrounding packaging.

I've been playing with meson and am having some success so far with pure python builds.

I am now playing around with swapping out backends using PDM.

See minimalist example here

In the above repo python -m build works well (especially after i installed ninja i can also use pip install --no-build-isolation .). I cant seem to get pdm build to work correctly.

is this a question i should be asking the PDM dev instead? the output below shows the error which seems to be it can't find the SDist in the dist directory. i just am not sure how to best approach using these tools together.
Many thanks for any direction.
i'm happy to post elsewhere if this is inappropriate here.

➜ pdm build -v           
Building sdist...
pdm.termui: Preparing isolated env for PEP 517 build...
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   meson-python
pdm.termui:   python>=3.9.16,<3.9.17
pdm.termui:   Adding requirement meson-python
pdm.termui:   Adding requirement python>=3.9.16,<3.9.17
pdm.termui: ======== Starting round 0 ========
pdm.termui: Pinning: python None
pdm.termui: ======== Ending round 0 ========
pdm.termui: ======== Starting round 1 ========
pdm.termui:   Adding requirement meson>=0.63.3(from meson-python 0.12.0)
pdm.termui:   Adding requirement pyproject-metadata>=0.6.1(from meson-python 0.12.0)
pdm.termui:   Adding requirement tomli>=1.0.0; python_version < "3.11"(from meson-python 0.12.0)
pdm.termui:   Adding requirement typing-extensions>=3.7.4; python_version < "3.10"(from meson-python 0.12.0)
pdm.termui: Pinning: meson-python 0.12.0
pdm.termui: ======== Ending round 1 ========
pdm.termui: ======== Starting round 2 ========
pdm.termui: Pinning: meson 1.0.0
pdm.termui: ======== Ending round 2 ========
pdm.termui: ======== Starting round 3 ========
pdm.termui:   Adding requirement packaging>=22.0(from pyproject-metadata 0.7.0)
pdm.termui: Pinning: pyproject-metadata 0.7.0
pdm.termui: ======== Ending round 3 ========
pdm.termui: ======== Starting round 4 ========
pdm.termui: Pinning: tomli 2.0.1
pdm.termui: ======== Ending round 4 ========
pdm.termui: ======== Starting round 5 ========
pdm.termui: Pinning: typing-extensions 4.4.0
pdm.termui: ======== Ending round 5 ========
pdm.termui: ======== Starting round 6 ========
pdm.termui: Pinning: packaging 23.0
pdm.termui: ======== Ending round 6 ========
pdm.termui: ======== Starting round 7 ========
pdm.termui: ======== Resolution Result ========
pdm.termui: Stable pins:
pdm.termui:               python None
pdm.termui:         meson-python 0.12.0
pdm.termui:                meson 1.0.0
pdm.termui:   pyproject-metadata 0.7.0
pdm.termui:                tomli 2.0.1
pdm.termui:    typing-extensions 4.4.0
pdm.termui:            packaging 23.0
pdm.termui: Installing meson-python 0.12.0
unearth.preparer: Using cached <Link https://files.pythonhosted.org/packages/ee/22/da1cc8cafca80283c795ebf58d4218017225df8288d70cb8fc32eb46f5e0/meson_python-0.12.0-py3-none-any.whl (from https://pypi.org/simple/meson-python/)>
pdm.termui: Installing meson 1.0.0
unearth.preparer: Using cached <Link https://files.pythonhosted.org/packages/2b/8c/694a2dda825ae6157fe0780bf4fbb1fc780cb7905bca7a8d0db87e0f50bc/meson-1.0.0-py3-none-any.whl (from https://pypi.org/simple/meson/)>
pdm.termui: Installing pyproject-metadata 0.7.0
unearth.preparer: Using cached <Link https://files.pythonhosted.org/packages/60/85/7de60a21931341f2b4907e4886f34f4c64cd19a28507b6940f284a857b10/pyproject_metadata-0.7.0-py3-none-any.whl (from https://pypi.org/simple/pyproject-metadata/)>
pdm.termui: Installing tomli 2.0.1
unearth.preparer: Using cached <Link https://files.pythonhosted.org/packages/97/75/10a9ebee3fd790d20926a90a2547f0bf78f371b2f13aa822c759680ca7b9/tomli-2.0.1-py3-none-any.whl (from https://pypi.org/simple/tomli/)>
pdm.termui: Installing typing-extensions 4.4.0
unearth.preparer: Using cached <Link https://files.pythonhosted.org/packages/0b/8e/f1a0a5a76cfef77e1eb6004cb49e5f8d72634da638420b9ea492ce8305e8/typing_extensions-4.4.0-py3-none-any.whl (from https://pypi.org/simple/typing-extensions/)>
pdm.termui: Installing packaging 23.0
unearth.preparer: Using cached <Link https://files.pythonhosted.org/packages/ed/35/a31aed2993e398f6b09a790a181a7927eb14610ee8bbf02dc14d31677f1c/packaging-23.0-py3-none-any.whl (from https://pypi.org/simple/packaging/)>
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   ninja>=1.8.2
pdm.termui:   python>=3.9.16,<3.9.17
pdm.termui:   Adding requirement ninja>=1.8.2
pdm.termui:   Adding requirement python>=3.9.16,<3.9.17
pdm.termui: ======== Starting round 0 ========
pdm.termui: Pinning: python None
pdm.termui: ======== Ending round 0 ========
pdm.termui: ======== Starting round 1 ========
pdm.termui: Pinning: ninja 1.11.1
pdm.termui: ======== Ending round 1 ========
pdm.termui: ======== Starting round 2 ========
pdm.termui: ======== Resolution Result ========
pdm.termui: Stable pins:
pdm.termui:   python None
pdm.termui:    ninja 1.11.1
pdm.termui: Installing ninja 1.11.1
unearth.preparer: Using cached <Link https://files.pythonhosted.org/packages/7c/61/5b9cd921db9f5efdea05422cb45a48a9ad22199f0ae566e7eda2174f384d/ninja-1.11.1-py2.py3-none-macosx_10_9_universal2.macosx_10_9_x86_64.macosx_11_0_arm64.macosx_11_0_universal2.whl (from https://pypi.org/simple/ninja/)>
pdm.termui: The Meson build system
pdm.termui: Version: 1.0.0
pdm.termui: Source dir: /Users/leahawasser/Documents/GitHub/pyos/examplePy/example6_pdm_meson
pdm.termui: Build dir: /Users/leahawasser/Documents/GitHub/pyos/examplePy/example6_pdm_meson/.mesonpy-uo73dwcl/build
pdm.termui: Build type: native build
pdm.termui: Project name: examplePy
pdm.termui: Project version: 1.0.0
pdm.termui: Host machine cpu family: aarch64
pdm.termui: Host machine cpu: aarch64
pdm.termui: Program python found: YES (/Users/leahawasser/opt/miniconda3/envs/pdm/bin/python3.9)
pdm.termui: Build targets in project: 0
pdm.termui: 
pdm.termui: examplePy 1.0.0
pdm.termui: 
pdm.termui:   User defined options
pdm.termui:     Native files: /Users/leahawasser/Documents/GitHub/pyos/examplePy/example6_pdm_meson/.mesonpy-native-file.ini
pdm.termui:     debug       : false
pdm.termui:     optimization: 2
pdm.termui:     prefix      : /Users/leahawasser/opt/miniconda3/envs/pdm
pdm.termui: 
pdm.termui: Found ninja-1.11.1.git.kitware.jobserver-1 at /var/folders/r8/3vljpqb55psbgb1ghc2qsn700000gn/T/pdm-build-env-8j1htas2-overlay/bin/ninja
pdm.termui: WARNING: Repository has uncommitted changes that will not be included in the dist tarball
pdm.termui: Created /Users/leahawasser/Documents/GitHub/pyos/examplePy/example6_pdm_meson/.mesonpy-uo73dwcl/build/meson-dist/examplePy-1.0.0.tar.gz
pdm.termui: + meson setup --prefix=/Users/leahawasser/opt/miniconda3/envs/pdm /Users/leahawasser/Documents/GitHub/pyos/examplePy/example6_pdm_meson /Users/leahawasser/Documents/GitHub/pyos/examplePy/example6_pdm_meson/.mesonpy-uo73dwcl/build --native-file=/Users/leahawasser/Documents/GitHub/pyos/examplePy/example6_pdm_meson/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2
pdm.termui: + meson dist --allow-dirty --no-tests --formats gztar
pdm.termui: Traceback (most recent call last):
pdm.termui:   File "/Users/leahawasser/opt/miniconda3/lib/python3.9/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
pdm.termui:     main()
pdm.termui:   File "/Users/leahawasser/opt/miniconda3/lib/python3.9/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
pdm.termui:     json_out['return_val'] = hook(**hook_input['kwargs'])
pdm.termui:   File "/Users/leahawasser/opt/miniconda3/lib/python3.9/site-packages/pyproject_hooks/_in_process/_in_process.py", line 304, in build_sdist
pdm.termui:     return backend.build_sdist(sdist_directory, config_settings)
pdm.termui:   File "/var/folders/r8/3vljpqb55psbgb1ghc2qsn700000gn/T/pdm-build-env-xfhk1f3c-shared/lib/python3.9/site-packages/mesonpy/__init__.py", line 1054, in wrapper
pdm.termui:     return func(*args, **kwargs)
pdm.termui:   File "/var/folders/r8/3vljpqb55psbgb1ghc2qsn700000gn/T/pdm-build-env-xfhk1f3c-shared/lib/python3.9/site-packages/mesonpy/__init__.py", line 1077, in build_sdist
pdm.termui:     return project.sdist(out).name
pdm.termui:   File "/var/folders/r8/3vljpqb55psbgb1ghc2qsn700000gn/T/pdm-build-env-xfhk1f3c-shared/lib/python3.9/site-packages/mesonpy/__init__.py", line 900, in sdist
pdm.termui:     with tarfile.open(meson_dist_path, 'r:gz') as meson_dist, mesonpy._util.create_targz(sdist) as (tar, mtime):
pdm.termui:   File "/Users/leahawasser/opt/miniconda3/envs/pdm/lib/python3.9/contextlib.py", line 119, in __enter__
pdm.termui:     return next(self.gen)
pdm.termui:   File "/var/folders/r8/3vljpqb55psbgb1ghc2qsn700000gn/T/pdm-build-env-xfhk1f3c-shared/lib/python3.9/site-packages/mesonpy/_util.py", line 49, in create_targz
pdm.termui:     file = typing.cast(IO[bytes], gzip.GzipFile(
pdm.termui:   File "/Users/leahawasser/opt/miniconda3/envs/pdm/lib/python3.9/gzip.py", line 173, in __init__
pdm.termui:     fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
pdm.termui: FileNotFoundError: [Errno 2] No such file or directory: '/Users/leahawasser/Documents/GitHub/pyos/examplePy/example6_pdm_meson/dist/examplepy-0.1.0.tar.gz'
Traceback (most recent call last):
  File "/Users/leahawasser/opt/miniconda3/bin/pdm", line 8, in <module>
    sys.exit(main())
  File "/Users/leahawasser/opt/miniconda3/lib/python3.9/site-packages/pdm/core.py", line 259, in main
    return Core().main(args)
  File "/Users/leahawasser/opt/miniconda3/lib/python3.9/site-packages/pdm/core.py", line 191, in main
    raise cast(Exception, err).with_traceback(traceback)
  File "/Users/leahawasser/opt/miniconda3/lib/python3.9/site-packages/pdm/core.py", line 186, in main
    f(project, options)
  File "/Users/leahawasser/opt/miniconda3/lib/python3.9/site-packages/pdm/cli/commands/build.py", line 66, in handle
    actions.do_build(
  File "/Users/leahawasser/opt/miniconda3/lib/python3.9/site-packages/pdm/cli/actions.py", line 506, in do_build
    loc = SdistBuilder(project.root, project.environment).build(
  File "/Users/leahawasser/opt/miniconda3/lib/python3.9/site-packages/pdm/builders/sdist.py", line 21, in build
    filename = self._hook.build_sdist(out_dir, config_settings)
  File "/Users/leahawasser/opt/miniconda3/lib/python3.9/site-packages/pyproject_hooks/_impl.py", line 292, in build_sdist
    return self._call_hook('build_sdist', {
  File "/Users/leahawasser/opt/miniconda3/lib/python3.9/site-packages/pyproject_hooks/_impl.py", line 311, in _call_hook
    self._subprocess_runner(
  File "/Users/leahawasser/opt/miniconda3/lib/python3.9/site-packages/pdm/builders/base.py", line 261, in subprocess_runner
    return log_subprocessor(cmd, cwd, extra_environ=env)
  File "/Users/leahawasser/opt/miniconda3/lib/python3.9/site-packages/pdm/builders/base.py", line 107, in log_subprocessor
    raise build_error(e) from None
pdm.exceptions.BuildError: Build backend raised error: Showing the last 10 lines of the build output:
    return project.sdist(out).name
  File "/var/folders/r8/3vljpqb55psbgb1ghc2qsn700000gn/T/pdm-build-env-xfhk1f3c-shared/lib/python3.9/site-packages/mesonpy/__init__.py", line 900, in sdist
    with tarfile.open(meson_dist_path, 'r:gz') as meson_dist, mesonpy._util.create_targz(sdist) as (tar, mtime):
  File "/Users/leahawasser/opt/miniconda3/envs/pdm/lib/python3.9/contextlib.py", line 119, in __enter__
    return next(self.gen)
  File "/var/folders/r8/3vljpqb55psbgb1ghc2qsn700000gn/T/pdm-build-env-xfhk1f3c-shared/lib/python3.9/site-packages/mesonpy/_util.py", line 49, in create_targz
    file = typing.cast(IO[bytes], gzip.GzipFile(
  File "/Users/leahawasser/opt/miniconda3/envs/pdm/lib/python3.9/gzip.py", line 173, in __init__
    fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
FileNotFoundError: [Errno 2] No such file or directory: '/Users/leahawasser/Documents/GitHub/pyos/examplePy/example6_pdm_meson/dist/examplepy-0.1.0.tar.gz'
You must be logged in to vote

I debugged this and I think it's 3 different bugs combined:

  • the PEP 517 specification doesn't say who creates the output directory
  • pdm doesn't create it, but does delete it if it doesn't exist
  • meson-python assumes that it was already created because build does

#274

Replies: 3 comments · 5 replies

Comment options

I debugged this and I think it's 3 different bugs combined:

  • the PEP 517 specification doesn't say who creates the output directory
  • pdm doesn't create it, but does delete it if it doesn't exist
  • meson-python assumes that it was already created because build does

#274

You must be logged in to vote
2 replies
@eli-schwartz
Comment options

I also opened a PDM bug report: pdm-project/pdm#1647

@lwasser
Comment options

Wow - thank you @eli-schwartz and @FFY00 i'll try again later today!
I did notice that my dist/ dir was removed during the build process. i just wasn't sure how to know whether it was pdm, meson-python or a config issue on my end. I'll test against a local build of this repo.

These two tools playing well together will be a real win i think for the community if we document it well. (I am planning to do that!).

Answer selected by FFY00
Comment options

I have now merged the fix.

You must be logged in to vote
0 replies
Comment options

hi @eli-schwartz @FFY00 I have had some success with using pdm with meson-python on a basic build.
i'm using mkdir dist && pdm build --no-clean

I've run into another curious issue.

my branch where i'm working is here:

When i install a dependency with PDM, it installs examplePy in editable mode as expected.

All packages are synced to date, nothing to do.
Installing the project as an editable package...
  ✔ Install examplePy 0.1.00 successful

But what is odd is when i use
conda list it lists the package as unknown

$ conda list
...
unknown                   0.0.0                    pypi_0    pypi

I see this same behavior in the python console when i print out a list of packages.

>>> import pkg_resources
>>> installed_packages = pkg_resources.working_set
>>> installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
...    for i in installed_packages])
>>> print(installed_packages_list)
[... 'meson-python==0.12.0', 'meson==1.0.0',  'packaging==23.0', 'pdm-pep517==1.0.6', 'pdm==2.4.0', 'pep517==0.13.0', 'pip==22.3.1',... 'requests==2.28.2', 'resolvelib==0.9.0', 'rich==13.3.0', 'setuptools==65.6.3', ... 'unknown==0.0.0']

So finally i thought.- perhaps i can just do a simple pip editable install. this doesn't work.
but pip install -e . does work for other pure python packages without a setup.py or cfg file.

$ pip install -e .
ERROR: Project file:///Users/leahawasser/Documents/GitHub/pyos/examplePy/example6_pdm_meson has a 'pyproject.toml' and its build backend is missing the 'build_editable' hook. Since it does not have a 'setup.py' nor a 'setup.cfg', it cannot be installed in editable mode. Consider using a build backend that supports PEP 660.

What does build however is this:

pip install --no-build-isolation .

When i install the package that way, i see it in the conda list output correctly - examplePy is listed with the correct version. And i can check it in my python console when i list packages and confirm it is there.

i am in the correct environment in python too:

>>> import os
>>> print(os.environ['CONDA_DEFAULT_ENV'])

but the package won't import.

>>> print(installed_packages_list)
[... 'distlib==0.3.6', 'examplepy==0.1.0', ... 'meson-python==0.12.0', 'meson==1.0.0', 'msgpack==1.0.3', 'packaging==23.0', 'pdm-pep517==1.0.6', 'pdm==2.4.0', 'pep517==0.13.0', 'pip==22.3.1',...]
>>> import examplepy # just in case something is case sensitive in a way i don't know it is 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'examplepy'
>>> import examplePy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'examplePy'
>>> 
>>> import os
>>> print(os.environ['CONDA_DEFAULT_ENV'])
pdm_meson

i am not sure here whether this is PDM, meson-python or me just not understanding how the build should work. the goal here is installing my package in editable mode.

You must be logged in to vote
3 replies
@eli-schwartz
Comment options

Meson-python does support editable mode -- every build backend needs to explicitly add support for it, including setup.py, but setup.py explicitly added support for it years and years and years ago.

Meson-python's support is not yet released though so you need to install it from git before you can use it. You'll have to ask @FFY00 when the next release is going to happen.

@FFY00
Comment options

FFY00 Jan 27, 2023
Collaborator

I'll do a pre-release soon, probably monday, with the initial support for editable installs.

@lwasser
Comment options

ok thank you both! i'll keep an eye on this repo. also it looks like a PDM file was merged and a new release was published yesterday! i really appreciate the help on all of this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
🙏
Q&A
Labels
None yet
3 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.