I noticed that the behaviour of the TestClient is stated wrong in the docs:
http://symfony.com/doc/current/book/testing.html#redirecting

The default behaviour of the TestClient is that it doesn't redirect. One has to first use $client->followRedirects() to get the client to follow all redirects.

If someone could confirm this issue I will rewrite the section!