EHN Add transform_inverse to Nystroem #19971
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kstoneriv3
commented
Apr 24, 2021
kstoneriv3
commented
Apr 24, 2021
sklearn/kernel_approximation.py
Outdated
| @@ -778,11 +836,17 @@ def fit(self, X, y=None): | ||
| **self._get_kernel_params()) | ||
|
|
||
| # sqrt of kernel matrix on basis vectors | ||
| U, S, V = svd(basis_kernel) | ||
| U, S, V = svd(basis_kernel) # TODO(kstoneriv3): Why not np.linalg.eigh() ? |
There was a problem hiding this comment.
I suppose eigh is faster than svd as the former is tuned specifically for Hermitian matrices. There is a comparison in the page below.
https://stackoverflow.com/questions/50358310/how-does-numpy-linalg-eigh-vs-numpy-linalg-svd
|
cc @ogrisel @lorentzenchr maybe? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reference Issues/PRs
As discussed in
#19732#19899,sklearn.kernel_approximation.Nystroemcurrently lackstransform_inverse()method unlikePCAandKernelPCA.What does this implement/fix? Explain your changes.
This PR implements
transform_inverseinsklearn.kernel_approximation.Nystroem.Any other comments?
While implementing the method, I was not confident about the choice of linear system solver APIs. Maybe there is a better API for solving systems with a positive semidefinite matrix.
The formula I used for kernel ridge regression of original data points to the low-dimensional representation vectors are as follows (it is also commented in the code).
The reconstruction performance was compared with KernelPCA using the example in #19945. The reconstruction quality seems identical in this case.
