Closed
Description
Problem
Thanks to the work of @nicolas-grekas and the community, Symfony tests are faster every day. But we can do better.
I've analyzed the Symfony 2.3 test suite and I've found that some tests are very slow. I can think of 3 reasons for a slow test:
- It's slow because the tested Symfony feature is slow. Let's try to fix that feature.
- It's slow because the test code is not good enough. Let's fix the test.
- It's slow because the test or the Symfony feature are complicated. Nothing is wrong with the test. Don't change anything.
Solution
- I propose to work on fixing the slowest Symfony tests.
- If you are experienced with Symfony, add a comment in this issue saying which tests are you going to working on, so we don't duplicate the work (e.g. I'm looking into Process component tests).
- If you are new to Symfony and want to contribute, you can ask me for help or guidance.
When fixing a test/feature to make it faster, keep in mind that we'll reject:
- Meaningless changes (e.g. replace single quotes by double quotes).
- Changes that destroy code readability.
In case of doubt, ask us!
Details
I took the fastest test as reference (in my machine it was InputArgumentTest::testGetDefault) and this is the list of the slowest Symfony tests:
Tests which are at least 10,000 times slower than the fastest test
21,107x slower : Symfony\Component\HttpFoundation\Tests\Session\Storage\MetadataBagTest::testInitialize
20,186x slower : Symfony\Component\Routing\Tests\Generator\Dumper\PhpGeneratorDumperTest::testDumpWithTooManyRoutes
12,647x slower : Symfony\Component\Process\Tests\ProcessTest::testProcessPipes with data set #9
[WORKING ON THIS] 10,754x slower : Symfony\Component\Filesystem\Tests\FilesystemTest::testCopyForOriginUrlsAndExistingLocalFileDefaultsToNotCopy
10,601x slower : Symfony\Component\Finder\Tests\Iterator\SortableIteratorTest::testAccept with data set #2
10,570x slower : Symfony\Component\Finder\Tests\Iterator\SortableIteratorTest::testAccept with data set #4
10,569x slower : Symfony\Component\Finder\Tests\Iterator\SortableIteratorTest::testAccept with data set #3
Tests which are at least 1,000 times slower than the fastest test
9,453x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\SecurityRoutingIntegrationTest::testRoutingErrorIsExposedWhenNotProtected with data set #0 ('config.yml')
9,348x slower : Symfony\Component\Finder\Tests\Iterator\RecursiveDirectoryIteratorTest::testRewindOnFtp
8,898x slower : Symfony\Component\Finder\Tests\Iterator\RecursiveDirectoryIteratorTest::testSeekOnFtp
8,420x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\SecurityRoutingIntegrationTest::testRoutingErrorIsNotExposedForProtectedResourceWhenLoggedInWithInsufficientRights with data set #1 ('routes_as_path.yml')
8,154x slower : Symfony\Bundle\FrameworkBundle\Tests\Functional\SessionTest::testTwoClients with data set #0 ('config.yml', true)
7,588x slower : Symfony\Component\Process\Tests\ProcessTest::testProcessPipes with data set #8 ('fwrite(STDOUT, $in = file_get... $in);', 4096)
7,074x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\SecurityRoutingIntegrationTest::testRoutingErrorIsNotExposedForProtectedResourceWhenLoggedInWithInsufficientRights with data set #0 ('config.yml')
7,064x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\SecurityRoutingIntegrationTest::testSecurityConfigurationForMultipleIPAddresses with data set #0 ('config.yml')
6,997x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\SecurityRoutingIntegrationTest::testSecurityConfigurationForMultipleIPAddresses with data set #1 ('routes_as_path.yml')
6,838x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\SecurityRoutingIntegrationTest::testRoutingErrorIsExposedWhenNotProtected with data set #1 ('routes_as_path.yml')
6,774x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\CsrfFormLoginTest::testFormLoginAndLogoutWithCsrfTokens with data set #0 ('config.yml')
6,014x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\FormLoginTest::testFormLogin with data set #1 ('routes_as_path.yml')
5,859x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\LocalizedRoutesAsPathTest::testLoginLogoutProcedure with data set #0 ('en')
5,800x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\LocalizedRoutesAsPathTest::testLoginFailureWithLocalizedFailurePath with data set #0 ('en')
5,646x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\SwitchUserTest::testSwitchedUserCannotSwitchToOther
4,590x slower : Symfony\Bundle\FrameworkBundle\Tests\Functional\SessionTest::testWelcome with data set #0 ('config.yml', true)
4,276x slower : Symfony\Component\Stopwatch\Tests\StopwatchEventTest::testDuration
4,142x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\FormLoginTest::testFormLogin with data set #0 ('config.yml')
4,141x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\CsrfFormLoginTest::testFormLoginAndLogoutWithCsrfTokens with data set #1 ('routes_as_path.yml')
3,977x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\LocalizedRoutesAsPathTest::testAccessRestrictedResourceWithForward with data set #0 ('en')
3,941x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\SecurityRoutingIntegrationTest::testRoutingErrorIsNotExposedForProtectedResourceWhenAnonymous with data set #1 ('routes_as_path.yml')
3,874x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\SwitchUserTest::testSwitchUser with data set "authorized_user_cannot_switch_to_non_existent" ('user_can_switch', 'user_does_not_exist', 'user_can_switch', 500)
3,849x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\SecurityRoutingIntegrationTest::testRoutingErrorIsNotExposedForProtectedResourceWhenAnonymous with data set #0 ('config.yml')
3,804x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\AuthenticationCommencingTest::testAuthenticationIsCommencingIfAccessDeniedExceptionIsWrapped
3,690x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\SecurityRoutingIntegrationTest::testSecurityConfigurationForSingleIPAddress with data set #0 ('config.yml')
3,686x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\SecurityRoutingIntegrationTest::testSecurityConfigurationForSingleIPAddress with data set #1 ('routes_as_path.yml')
3,332x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\FirewallEntryPointTest::testItUsesTheConfiguredEntryPointWhenUsingUnknownCredentials
3,278x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\FirewallEntryPointTest::testItUsesTheConfiguredEntryPointFromTheExceptionListenerWithFormLoginAndNoCredentials
3,116x slower : Symfony\Component\Process\Tests\ProcessTest::testGetExitCodeIsNullOnWhenStartingAgain
3,006x slower : Symfony\Bundle\FrameworkBundle\Tests\Functional\ProfilerTest::testProfilerIsDisabled with data set #1 (true)
2,816x slower : Symfony\Bundle\FrameworkBundle\Tests\Functional\SessionTest::testFlash with data set #0 ('config.yml', true)
2,323x slower : Symfony\Bundle\FrameworkBundle\Tests\Command\CacheClearCommand\CacheClearCommandTest::testCacheIsFreshAfterCacheClearedWithWarmup
2,278x slower : Symfony\Component\Process\Tests\ProcessTest::testStartAfterATimeout
2,214x slower : Symfony\Component\Stopwatch\Tests\StopwatchEventTest::testEnsureStopped
2,176x slower : Symfony\Component\Process\Tests\ProcessTest::testRunProcessWithTimeout
2,166x slower : Symfony\Component\Stopwatch\Tests\StopwatchTest::testStop
2,005x slower : Symfony\Bundle\SecurityBundle\Tests\DependencyInjection\YamlCompleteConfigurationTest::testEncoders
1,991x slower : Symfony\Bundle\SecurityBundle\Tests\Functional\CsrfFormLoginTest::testFormLoginWithInvalidCsrfToken with data set #1 ('routes_as_path.yml')
1,938x slower : Symfony\Bundle\FrameworkBundle\Tests\Templating\Helper\FormHelperDivLayoutTest::testRest
1,848x slower : Symfony\Bridge\Twig\Tests\Extension\FormExtensionDivLayoutTest::testThemeBlockInheritanceUsingUse
[IMPROVED] 1,784x slower : Symfony\Bundle\FrameworkBundle\Tests\DependencyInjection\XmlFrameworkExtensionTest::testDisabledProfiler
[IMPROVED] 1,758x slower : Symfony\Bundle\FrameworkBundle\Tests\DependencyInjection\YamlFrameworkExtensionTest::testValidation
[IMPROVED] 1,736x slower : Symfony\Bundle\FrameworkBundle\Tests\DependencyInjection\XmlFrameworkExtensionTest::testValidation
1,732x slower : Symfony\Bundle\FrameworkBundle\Tests\DependencyInjection\YamlFrameworkExtensionTest::testEsi
[IMPROVED] 1,703x slower : Symfony\Bundle\FrameworkBundle\Tests\DependencyInjection\PhpFrameworkExtensionTest::testValidation
[IMPROVED] 1,680x slower : Symfony\Bundle\FrameworkBundle\Tests\DependencyInjection\PhpFrameworkExtensionTest::testEsi
1,658x slower : Symfony\Bundle\SecurityBundle\Tests\DependencyInjection\XmlCompleteConfigurationTest::testEncoders
1,617x slower : Symfony\Component\HttpKernel\Tests\HttpCache\HttpCacheTest::testDoesNotCacheSomeStatusCodeResponses
1,614x slower : Symfony\Bundle\SecurityBundle\Tests\DependencyInjection\PhpCompleteConfigurationTest::testRolesHierarchy
1,604x slower : Symfony\Component\Process\Tests\ProcessTest::testGetExitCodeIsNullOnStart
1,587x slower : Symfony\Component\Process\Tests\ProcessTest::testIsSuccessfulOnlyAfterTerminated
1,574x slower : Symfony\Bundle\SecurityBundle\Tests\DependencyInjection\SecurityExtensionTest::testFirewallWithoutAuthenticationListener
1,560x slower : Symfony\Component\Process\Tests\ProcessTest::testStatus
1,536x slower : Symfony\Component\Process\Tests\ProcessTest::testTTYCommand
1,462x slower : Symfony\Component\Process\Tests\ProcessTest::testStopWithTimeoutIsActuallyWorking
1,377x slower : Symfony\Bundle\FrameworkBundle\Tests\Templating\Helper\FormHelperDivLayoutTest::testDateTime
1,358x slower : Symfony\Bundle\FrameworkBundle\Tests\Templating\Helper\FormHelperTableLayoutTest::testSingleChoiceRequiredWithEmptyValueViaView
1,329x slower : Symfony\Bundle\FrameworkBundle\Tests\Templating\Helper\FormHelperTableLayoutTest::testMoney
1,285x slower : Symfony\Bundle\FrameworkBundle\Tests\Templating\Helper\FormHelperDivLayoutTest::testTimeWithEmptyValueGlobal
1,260x slower : Symfony\Component\Form\Tests\Extension\Validator\Type\FormTypeValidatorExtensionTest::testValidationGroupsCanBeSetToCallback
1,253x slower : Symfony\Bridge\ProxyManager\Tests\LazyProxy\PhpDumper\ProxyDumperTest::testGetProxyCode
1,227x slower : Symfony\Bundle\FrameworkBundle\Tests\DependencyInjection\ConfigurationTest::testValidTrustedProxies with data set #3 (null, array())
1,174x slower : Symfony\Component\Console\Tests\Helper\DialogHelperTest::testAskWithAutocomplete
1,146x slower : Symfony\Component\Process\Tests\ProcessTest::testCheckTimeoutOnStartedProcess
1,119x slower : Symfony\Component\BrowserKit\Tests\ClientTest::testInsulatedRequests
1,117x slower : Symfony\Component\Validator\Tests\Constraints\CardSchemeValidatorTest::testInvalidNumbers with data set #0 ('VISA', '42424242424242424242')
1,106x slower : Symfony\Component\Finder\Tests\BsdFinderTest::testName
1,083x slower : Symfony\Component\ClassLoader\Tests\ClassCollectionLoaderTest::testCommentStripping
1,074x slower : Symfony\Bundle\FrameworkBundle\Tests\Functional\FragmentTest::testFragment with data set #1 (true)
1,073x slower : Symfony\Component\Stopwatch\Tests\StopwatchEventTest::testStartTime
1,065x slower : Symfony\Component\Process\Tests\ProcessTest::testRestart
1,061x slower : Symfony\Component\HttpFoundation\Tests\File\FileTest::testMove
1,014x slower : Symfony\Component\Intl\Tests\NumberFormatter\Verification\NumberFormatterTest::testFormatRoundingModeHalfEven with data set #0 (1.121, '1.12')
1,007x slower : Symfony\Component\Form\Tests\CompoundFormTest::testSetDataSupportsDynamicAdditionAndRemovalOfChildren
Metadata
Metadata
Assignees
Labels
No labels