Pull Request Tests with [feature] tag in commits should report failed in CI when "Feature" tests fail #4641
Description
Metadata
Metadata
Assignees
Labels
The issue is fixed.The issue is fixed.issues in a test or in test infrastructureissues in a test or in test infrastructure
Type
Fields
Give feedbackNo fields configured for issues without a type.
I have had 2 pull requests merged where I missed the failing tests due to Travis CI reporting the build passing when tests actually failed. It is partially my fault for not being completely attentive to the test results, but in my defense there are a very large number of feature tests run and it is easy to lose track in the sea of results.
IMO, the CI should report the build failed when when the [feature] tag is present in the commit and any "Feature" test fails. This will make it easier to spot when something goes wrong.
Example PR #4494 Travis CI Passing with failed test https://travis-ci.org/PowerShell/PowerShell/jobs/266123025