Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

PERF Force OpenBLAS to use an OpenMP backend #29403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: main
Choose a base branch
Loading
from

Conversation

jeremiedbb
Copy link
Member

closes #28883

Here's an attempt to use the incoming OpenBLAS feature allowing to register a user threading callback. The goal is to make OpenBLAS use scikit-learn's OpenMP threadpool instead of its own. See the linked issue for more details about the benefits.

The PR is not ready for merge:

  • this feature will be available in OpenBLAS 0.3.28 which is not released yet. I'm testing it locally for now.
  • this feature has issues that may or may not be solvable because of the way we handle flexible OpenMP threadpool sizes (see Thread callback for OpenMP backend OpenMathLib/OpenBLAS#4770)
  • there's no way to make this callback use a dynamically defined threadpool size.
  • it has a side effect on the whole program, not just within scikit-learn, because it register the callback on all linked openblas libs, which can be used by other libraries. It would be possible to make a context manager that unregister the callback on exit. I don't see us doing that everywhere though.
  • registering the callback at import time as done in this PR for now should be avoided because of the previous point and because it requires to instantiate the threadpool controller at import time which we tried and caused a deadlock in 1.5.0.

Copy link

github-actions bot commented Jul 3, 2024

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

Generated for commit: b104d5e. Link to the linter CI: here

@jeremiedbb jeremiedbb changed the title ENH Force OpenBLAS to use an OpenMP backend PERF Force OpenBLAS to use an OpenMP backend Jul 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Configure OpenBLAS to use scikit-learn's OpenMP threadpool
1 participant
Morty Proxy This is a proxified and sanitized view of the page, visit original site.