Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Introduce RetryAttribute for test methods - #4586

#4586
Merged
Evangelink merged 9 commits into
microsoft:mainmicrosoft/testfx:mainfrom
Youssef1313:retry-attrYoussef1313/testfx:retry-attrCopy head branch name to clipboard
Jan 15, 2025
Merged

Introduce RetryAttribute for test methods#4586
Evangelink merged 9 commits into
microsoft:mainmicrosoft/testfx:mainfrom
Youssef1313:retry-attrYoussef1313/testfx:retry-attrCopy head branch name to clipboard

Conversation

@Youssef1313

Copy link
Copy Markdown
Member

Fixes #3161

Comment thread src/TestFramework/TestFramework/Attributes/TestMethod/RetryAttribute.cs Outdated
Comment thread src/TestFramework/TestFramework/Attributes/TestMethod/RetryAttribute.cs Outdated
Comment thread src/TestFramework/TestFramework/Attributes/TestMethod/RetryAttribute.cs Outdated
Comment thread src/TestFramework/TestFramework/Attributes/TestMethod/RetryAttribute.cs Outdated
Comment thread src/TestFramework/TestFramework/Attributes/TestMethod/RetryAttribute.cs Outdated
Comment thread src/Adapter/MSTest.TestAdapter/Execution/TestMethodInfo.cs
@nohwnd

nohwnd commented Jan 9, 2025

Copy link
Copy Markdown
Member

FYI From previous talks with Playwright where they have good retries, that imho were attempted to be duplicated in mstest.playwright, Max also mentioned that they wanted:

  • max retries to be configurable via config
  • see what max retries value is in assembly initialize (might be specific to them because they are in position of test framework in vstest, so that story might not be relevant here, where they have way more power via extensions easily)
  • have good reporting of the retries, to be able to show retries that were not passing

@Youssef1313

Youssef1313 commented Jan 9, 2025

Copy link
Copy Markdown
Member Author

max retries to be configurable via config

Not sure what shape would you be expecting. Currently, every application of the attribute is specifying the number of retries


see what max retries value is in assembly initialize (might be specific to them because they are in position of test framework in vstest, so that story might not be relevant here, where they have way more power via extensions easily)

This is currently info specific to the test method itself and is irrelevant to assembly initialize. I think you are talking of a completely different design that allows a "global" retry count, not a Retry attribute on test methods which I'm implementing here.


have good reporting of the retries, to be able to show retries that were not passing

We are going to have a design that allows that, but that is unlikely to be implemented in this PR though, unfortunately. We need to know better how to communicate that information to tools that will make use of it.

@Youssef1313
Youssef1313 force-pushed the retry-attr branch 7 times, most recently from aeccb48 to 077b03b Compare January 14, 2025 04:22
Comment thread src/Adapter/MSTest.TestAdapter/Execution/UnitTestRunner.cs
Comment thread src/Adapter/MSTest.TestAdapter/Execution/TestMethodInfo.cs Outdated
Evangelink
Evangelink previously approved these changes Jan 14, 2025
Comment thread src/Adapter/MSTest.TestAdapter/Execution/TestMethodInfo.cs Outdated
Comment thread test/IntegrationTests/MSTest.Acceptance.IntegrationTests/RetryTests.cs Outdated
Comment thread test/IntegrationTests/MSTest.Acceptance.IntegrationTests/RetryTests.cs Outdated
Comment thread test/IntegrationTests/MSTest.Acceptance.IntegrationTests/RetryTests.cs Outdated
Comment thread src/Adapter/MSTest.TestAdapter/Resources/Resource.resx Outdated
Comment thread src/TestFramework/TestFramework/Attributes/TestMethod/RetryBaseAttribute.cs Outdated
Comment thread src/TestFramework/TestFramework/Attributes/TestMethod/RetryContext.cs Outdated
@Evangelink
Evangelink merged commit c93d067 into microsoft:main Jan 15, 2025
@Youssef1313
Youssef1313 deleted the retry-attr branch January 15, 2025 19:34
Comment on lines +127 to +129
// This is a public API, so we can't change it to be async.
// Consider not using this API internally, and introduce an async version, and mark this as obsolete.
ExecuteTestsAsync(tests, runContext, frameworkHandle, isDeploymentDone).GetAwaiter().GetResult();

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I need to address that in a follow-up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Develop Retry Attribute for TestMethod

4 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.