Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

reimplementation of gpu_count #3718

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 19, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
reimplementation of gpu_count
  • Loading branch information
mauriliogenovese committed Mar 4, 2025
commit 4dfbbbee48fb92927dacb1905dc77bb5ed7382e4
2 changes: 1 addition & 1 deletion 2 nipype/pipeline/plugins/multiproc.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
from ...utils.profiler import get_system_total_memory_gb
from ..engine import MapNode
from .base import DistributedPluginBase
from .tools import gpu_count
from ...utils.gpu_count import gpu_count

try:
from textwrap import indent
Expand Down
10 changes: 0 additions & 10 deletions 10 nipype/pipeline/plugins/tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -175,13 +175,3 @@ def create_pyscript(node, updatehash=False, store_exception=True):
with open(pyscript, "w") as fp:
fp.writelines(cmdstr)
return pyscript


def gpu_count():
n_gpus = 1
try:
import GPUtil
except ImportError:
return 1
else:
return len(GPUtil.getGPUs())
55 changes: 55 additions & 0 deletions 55 nipype/utils/gpu_count.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# -*- DISCLAIMER: this file contains code derived from gputil (https://github.com/anderskm/gputil)
# and therefore is distributed under to the following license:
#
# MIT License
#
# Copyright (c) 2017 anderskm
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

import platform
import shutil
from subprocess import Popen, PIPE
import os


def gpu_count():
try:
if platform.system() == "Windows":
nvidia_smi = shutil.which('nvidia-smi')
if nvidia_smi is None:
nvidia_smi = (

Check warning on line 37 in nipype/utils/gpu_count.py

View check run for this annotation

Codecov / codecov/patch

nipype/utils/gpu_count.py#L35-L37

Added lines #L35 - L37 were not covered by tests
"%s\\Program Files\\NVIDIA Corporation\\NVSMI\\nvidia-smi.exe"
% os.environ['systemdrive']
)
else:
nvidia_smi = "nvidia-smi"

p = Popen(
[nvidia_smi, "--query-gpu=name", "--format=csv,noheader,nounits"],
stdout=PIPE,
)
stdout, stderror = p.communicate()

Check warning on line 48 in nipype/utils/gpu_count.py

View check run for this annotation

Codecov / codecov/patch

nipype/utils/gpu_count.py#L48

Added line #L48 was not covered by tests

output = stdout.decode('UTF-8')
lines = output.split(os.linesep)
num_devices = len(lines) - 1
return num_devices

Check warning on line 53 in nipype/utils/gpu_count.py

View check run for this annotation

Codecov / codecov/patch

nipype/utils/gpu_count.py#L50-L53

Added lines #L50 - L53 were not covered by tests
except:
return 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the goal is to keep it as close to the source as possible, that's fine. Here are some cleanups for your consideration:

  1. Use shutil.which() unconditionally. We can detect a FileNotFoundError before calling the process and simply return 0.
  2. Use subprocess.run, which is the recommended API. By using text mode, we don't have to handle decoding or newline normalization.
  3. Use targeted errors. FileNotFoundError and PermissionError (both OSErrors) should catch failures to run. UnicodeDecodeError will catch bad output.
  4. Use p.output.splitlines() to get content-full lines. A trailing newline does not produce an empty string at the end of the list.
Suggested change
from subprocess import Popen, PIPE
import os
def gpu_count():
try:
if platform.system() == "Windows":
nvidia_smi = shutil.which('nvidia-smi')
if nvidia_smi is None:
nvidia_smi = (
"%s\\Program Files\\NVIDIA Corporation\\NVSMI\\nvidia-smi.exe"
% os.environ['systemdrive']
)
else:
nvidia_smi = "nvidia-smi"
p = Popen(
[nvidia_smi, "--query-gpu=name", "--format=csv,noheader,nounits"],
stdout=PIPE,
)
stdout, stderror = p.communicate()
output = stdout.decode('UTF-8')
lines = output.split(os.linesep)
num_devices = len(lines) - 1
return num_devices
except:
return 0
import subprocess
import os
def gpu_count():
nvidia_smi = shutil.which('nvidia-smi')
if nvidia_smi is None and platform.system() == "Windows":
nvidia_smi = f'{os.environ["systemdrive"]}\\Program Files\\NVIDIA Corporation\\NVSMI\\nvidia-smi.exe'
if nvidia_smi is None:
return 0
try:
p = subprocess.run(
[nvidia_smi, "--query-gpu=name", "--format=csv,noheader,nounits"],
stdout=subprocess.PIPE,
text=True,
)
except (OSError, UnicodeDecodeError):
return 0
return len(output.splitlines())

Morty Proxy This is a proxified and sanitized view of the page, visit original site.