Can weights be negative? If so, I think this might be more appropriate:

cc @jeremiedbb which whom we discussed normalization with negative weights.

IIRC self.weights_ should be non-negative at this point. Not completely sure. I will try to take a look later, time permitting (though this isn't too high on my list)

Side comment: does anyone know of applications that are wide spread and use negative sample weights?

The only people I know who have negative sample weights are (some) researchers in High Energy Particle physics. From my memory the interpretation of what those samples with negative weights mean is tricky. The consensus I remember is that a group of samples should always have a net positive weight (but this isn't always true), but that you shouldn't take the absolute value of a individual sample's weight. The reason for not using abs(weight) is that a possible interpretation of a negative sample weight is that you are trying to "subtract" that sample from your final group of samples. (I think the "deep physics" reasons for how you end up with them is that quantum mechanics has constructive and destructive interference terms, and depending on how your sample generator is written you end up with samples with a negative weight that are meant to represent negative interference, or something like that.)

Basically, are negative weights something worth spending time on or best to punt on it till there are use-cases?

Basically, are negative weights something worth spending time on or best to punt on it till there are use-cases?

AFAIK not really, but I wonder what prompted @jjerphan and @jeremiedbb's conversations on normalizing negative weights.

What @betatim described prompted those discussions; I think @betatim definitely is more knowledgeable than me on this subject.

I think, use-cases relying on negative weights are relatively rare, and this might not be something we would like to maintain relevant normalization for if this normalization is complex.

I think it is good to assume that weights are always positive and use that to reduce complexity/additional computation.

As long as we avoid (re-normalizing) user provided weights_init (see my review below), with the current capabilities of Mixture classes like GMM, I don't see how its possible to get negative weights. Thus I hope this isn't something we have to consider one way or another.

And I would really push to not re-normalize user provided weights_init, and especially to not make rigid decisions on how to re-normalize negative vs. positive weights, etc. Otherwise we take away the ability to precisely define an initial state for the EM algorithm.

The negative weights come back all the time ;). The last report was the following: #18934

If really only want positive weight, we should use only_non_negative=True in _check_sample_weight. I assume that we should come up with a PR allowing negative weight for people that really want it but without any guarantee that the normalization makes any sense. If an algorithm was written to take into account negative weight then we can always make sure that the normalization is properly done as per the literature.

The normalization should not be done after the user input initialization! We expect normalized weights to be given in weights_init. It's previous location looks correct to me.

I suggest that, as with the other comment, the weights normalization is done here. We expect normalized weights to be input in weights_init.

As per the negative weights, by moving the normalization step here, we largely avoid this problem, as the current inputs shouldn't really provide negative weights as far as I am aware. (only applying normalization if we run the _estimate_gaussian_parameters step.), but not re-normalizing what might be carefully normalized user weights),

However, PR #24812 making GMM initialization more robust (allowing users to pass in their own responsibilities matrix, or to use a custom callable, such as a pre-fitted K-means) will make this matter. So it's worth considering the issue here.