Description:

Since a week or so, some commands that come from Python packages (e.g. coveralls) are no longer found in the PATH under some conditions (see below). Our test workflow errors out when it attempts to run the coverallscommand, with:

/Users/runner/work/_temp/43f872ff-0703-452b-952c-16732d678e6d.sh: line 1: coveralls: command not found

This happened in today's run of the test workflow (example for Python 3.11 on MacOS):
https://github.com/zhmcclient/python-zhmcclient/actions/runs/10865840451/job/30157793391

It did not yet happen in last week's run of the test workflow (example for Python 3.11 on MacOS):
https://github.com/zhmcclient/python-zhmcclient/actions/runs/10754763507/job/29825736678

It happens across all of our projects that invoke coveralls, and the issue does not go away when re-running the failed workflows.

I dug into this, and found the following:

The issue only occurs when all of the following conditions are met:

• MacOS is used
• Python 3.9, 3.10, or 3.11 is used
• The workflow task specifies a login shell via shell: bash -l {0}

The issue does not happen, for example:

• on Ubuntu or Windows
• on MacOS with Python 3.8 or 3.12
• on MacOS with Python 3.9, 3.10, 3.11 when no login shell is specified

I found two changes that caused this (the following applies to MacOS):

• Some Python related directories are not in the PATH (example for Python 3.11):

  • /Users/runner/hostedtoolcache/Python/3.11.9/arm64/bin
  • /Users/runner/hostedtoolcache/Python/3.11.9/arm64

  This occurs when a login shell is specified (for all Python versions 3.8-3.12), but not when no login shell is specified.

• The commands from Python packages are not installed in /Library/Frameworks/Python.framework/Versions/Current/bin/. That directory is always in the PATH (i.e. for all Python versions 3.8-3.12, and regardless of whether or not a login shell is specified). When all of the conditions above are met, that directory still contains commands like python, pip, idle3, etc, but it no longer contains any command that comes from a Python package, e.g. coveralls, coverage, flake8, etc.

The /Library/Frameworks/Python.framework/Versions/Current is a softlink to /Library/Frameworks/Python.framework/Versions/3.12 for all of Python 3.9, 3.10 and 3.11. I guess that 3.12 is the preinstalled Python version and setup-python simply leaves that one alone. So because /Users/runner/hostedtoolcache/Python/3.11.9/arm64/bin is not in the PATH, the only Python bin directory that is in the PATH is the preinstalled 3.12 and that one obviously does not contain the Python based commands that were installed into 3.11 (or 3.10 or 3.9). The "missing" Python based commands can be found in the correct Python version's directory at /Library/Frameworks/Python.framework/Versions/3.11/bin (for 3.11)

An interesting question is why it works for Python 3.8, but I did not dig into that further. Python 3.8 is installed differently than the later Python versions.

I have debug tasks in the test workflow that show all of the above for different Python versions and for Ubuntu and MacOS, for comparison. See here: https://github.com/zhmcclient/python-zhmcclient/actions/runs/10869252280

I do not know how all of this was a week ago when it still worked, unfortunately, but it did work back then.

We will be fixing the issue in our repos by no longer specifying a login shell when we invoke coveralls. I was looking into the commit history of several of our repos and it seems the use of the login shell was there from the very beginning, but just when invoking "coveralls". I do not understand why it was used in the first place, and I probably just picked that up somewhere, but back then there must have been a reason for it.

However, I still think it is very unexpected that the use of a login shell causes Python based commands no longer to be found, so this issue should still be addressed.

Action version:

actions/setup-python version: v5

Platform:

• Ubuntu
• macOS
• Windows

Runner type:

• Hosted
• Self-hosted

Tools version:

Affected are Python 3.9, 3.10, 3.11 on MacOS

Repro steps:

1. Have a workflow that does all of the below:

    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: 3.11
    - name: Install coveralls
      run: |
        pip install coveralls
    - name: Run coveralls without login shell (succeeds)
      run: |
        coveralls --version
    - name: Run coveralls with login shell (fails)
      shell: bash -l {0}
      run: |
        coveralls --version

1. Run the workflow

Expected behavior:

Python package based commands should be found regardless of whether a login shell is specified.

Actual behavior:

See above