Another odd thing about this failure is that http://hg.python.org/cpython/rev/59a11c81dd3c should have fixed it as it should be setting warnings.showwarning() back to it's original value before the test began executing. So I checked PyShell again and found out it's being very naughty.

If you look at http://hg.python.org/cpython/file/ec4b0145f7e5/Lib/idlelib/PyShell.py#l76 you will find that it is replacing warnings.formatwarning, and that's a big no-no. While warnings.showwarning is designed to be replaced as a hooking point (http://docs.python.org/3/library/warnings.html#warnings.showwarning), formatwarning is not and is meant to be the default function showwarning calls to produce the string that is to be used in the warning (http://docs.python.org/3/library/warnings.html#warnings.formatwarning).

My hunch (which I currently can't verify since I'm at work) is that if you stop replacing warnings.formatwarning and instead just have IDLE's showwarning version call its formatting function directly then this failure will go away. The warning about test_idle mucking with warnings.showwarning will still be there until a way to optionally set showwarning is developed, but at least the buildbots should go back to green.