Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

eliminate performance regression when normalize is False #19606

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 3, 2021

Conversation

maikia
Copy link
Contributor

@maikia maikia commented Mar 3, 2021

closes #19600

There was a major performance regression in the linear models. This was due to the new use of the _incremental_mean_and_var().
In the case when normalize parameter is not set in the linear model calculations of the variance are not necessary. This PR exchanges it for np.average() in case when normalize is set to False.

Performance (current main):

main

zoomed into _preprocess_data at _base.py:

main_zoom

Performance (this PR):

after

zoomed into _preprocess_data at _base.py:

after_zoom

The performance is measured using the code of @jeremiedbb :

from sklearn.linear_model import ElasticNet 
from asv_benchmarks.benchmarks.datasets import _synth_regression_dataset 
data = _synth_regression_dataset(n_samples=5000, n_features=10000) 
X, _, y, _ = data 
estimator = ElasticNet(precompute=False, alpha=100, random_state=0) 
%load_ext snakeviz 
%snakeviz estimator.fit(X, y)

cc @ogrisel @jeremiedbb @agramfort

@maikia maikia changed the title exchange _incremental_mean_and_var for np.averate when not normalize eliminate performance regression when normalize is False Mar 3, 2021
Copy link
Member

@jeremiedbb jeremiedbb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. Just 1 small thing

sklearn/linear_model/_base.py Outdated Show resolved Hide resolved
Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>
@jeremiedbb jeremiedbb merged commit 1045d16 into scikit-learn:main Mar 3, 2021
@jeremiedbb
Copy link
Member

Thanks @maika !

@ogrisel
Copy link
Member

ogrisel commented Mar 4, 2021

Thanks @maikia, the nighlty benchmark report confirms that the perf regression has been fixed:

https://scikit-learn.org/scikit-learn-benchmarks/#linear_model.ElasticNetBenchmark.time_fit?p-representation='dense'&p-precompute=False

@glemaitre glemaitre mentioned this pull request Apr 22, 2021
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Performance regression in linear models
4 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.