This issue is about agreeing on what to do about common tests for estimators that support the Array API.

There are two things we need to test for every estimator for which we add Array API support:

1. does it work with a selection of Array API implementations. I think testing with numpy.array_api, pytorch (CPU) as default and pytoch (GPU) and cupy when a GPU is available would be good and cover most bases.
2. do the results change when a Array API namespace is used compared to a vanilla Numpy array

There is a 2.5 point as well: different dtypes (float64, float32, float16 (when supported)). Not sure what to do about that in terms of common test.

is there something else we should test in the common tests?

Implementation wise I think it would be good to add a estimator tag that we can use to mark estimators that support Array API. Something like "array_api_support": True?

Then I'd add one test that checks if the result of vanilla Numpy match the results of numpy.array_api (to cover (2)). And then maybe two more common tests. One to check that numpy.array_api and pytorch (CPU) work and another one for the GPU based tests in (1)? I'm wondering if having some code duplication between tests for (1) is worth it in terms of making the tests easier to read compared to trying to do it all via parametrisation.

Depending on how far #26243, #26315, #22554, #25956 are we should also clean up the tests that check this that were added in those PRs.

What do you think @ogrisel @thomasjpfan ?