Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

FEA Store the GradientBoostingClassifier/ GradientBoostingRegressor OOB Scores as the oob_score(s)_ fitted attributes #24882

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 48 commits into from
Jan 12, 2023

Conversation

awinml
Copy link
Contributor

@awinml awinml commented Nov 10, 2022

Reference Issues/PRs

Fixes #23400

GradientBoostingClassifier and GradientBoostingRegressor now exposes out-of-bag scores via the oob_scores_ or oob_score_ attributes.

What does this implement/fix? Explain your changes.

Added oob_scores_ and oob_score_ attributes to Gradient Boosting Estimators to store the Out Of Bag loss.

  • oob_scores_ contains the full history of loss values.
  • oob_score_ stores the last element of oob_scores_ for easy access.

@awinml
Copy link
Contributor Author

awinml commented Nov 10, 2022

@ogrisel Would please kindly review this PR?

@awinml awinml changed the title FEAT Store the OOB Loss for GradientBoostingClassifier FEA Store the OOB Loss for GradientBoostingClassifier Nov 21, 2022
@jjerphan jjerphan changed the title FEA Store the OOB Loss for GradientBoostingClassifier FEA Store the GradientBoostingClassifier OOB Scores as the oob_score(s)_ fitted attributes Dec 1, 2022
Copy link
Member

@jjerphan jjerphan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this contribution, @awinml.

sklearn/ensemble/_gb.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
doc/whats_new/v1.2.rst Outdated Show resolved Hide resolved
@jjerphan jjerphan added Waiting for Second Reviewer First reviewer is done, need a second one! and removed cython labels Dec 1, 2022
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
doc/whats_new/v1.3.rst Outdated Show resolved Hide resolved
sklearn/ensemble/_gb.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
awinml and others added 8 commits December 2, 2022 12:56
Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Copy link
Member

@glemaitre glemaitre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another pass.

sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/_gb.py Outdated Show resolved Hide resolved
sklearn/ensemble/_gb.py Outdated Show resolved Hide resolved
@awinml awinml requested a review from glemaitre January 10, 2023 19:52
sklearn/ensemble/_gb.py Show resolved Hide resolved
Copy link
Member

@glemaitre glemaitre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we converge.

sklearn/ensemble/_gb.py Outdated Show resolved Hide resolved
sklearn/ensemble/_gb.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/_gb.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
@awinml awinml requested a review from glemaitre January 11, 2023 12:08
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
sklearn/ensemble/_gb.py Outdated Show resolved Hide resolved
Copy link
Member

@glemaitre glemaitre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the comments from @betatim are addressed, LGTM on my side.

awinml and others added 2 commits January 11, 2023 18:29
@awinml awinml changed the title FEA Store the GradientBoostingClassifier OOB Scores as the oob_score(s)_ fitted attributes FEA Store the GradientBoostingClassifier/ GradientBoostingRegressor OOB Scores as the oob_score(s)_ fitted attributes Jan 11, 2023
sklearn/ensemble/tests/test_gradient_boosting.py Outdated Show resolved Hide resolved
@jjerphan
Copy link
Member

Letting @glemaitre merge once this LGTH.

@glemaitre glemaitre merged commit c676917 into scikit-learn:main Jan 12, 2023
@glemaitre
Copy link
Member

Merging Thanks @awinml

glemaitre added a commit to glemaitre/scikit-learn that referenced this pull request Jan 12, 2023
…ed attributes (scikit-learn#24882)

Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Guillaume Lemaitre <g.lemaitre58@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Store the OOB Loss for GradientBoostingClassifier
4 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.