After

rcParams["xtick.direction"] = rcParams["ytick.direction"] = "out"
xlim(4.001, 8)

There is an extra label at x=4.0, but (correctly) no tick there:

Probably a rounding issue, e.g. it doesn't appear using xlim(4.005, 8). I understand that some tolerance in floating point comparison is needed, but at least the same should be used for both ticks and labels (i.e. I wouldn't necessarily consider it a bug if both a tick and a label appeared when the lower xlim is e.g. 4+1e-10).

Sadly not solved by #5768.

PS: I guess this is going to be much more visible now that the default style has ticks pointing outwards...