Automatic perf stats in baselines#57730
Merged
RyanCavanaugh merged 10 commits intoMar 15, 2024
microsoft:mainmicrosoft/TypeScript:mainfrom
RyanCavanaugh:perf-baseline-statsRyanCavanaugh/TypeScript:perf-baseline-statsCopy head branch name to clipboard
Merged
Automatic perf stats in baselines#57730RyanCavanaugh merged 10 commits intomicrosoft:mainmicrosoft/TypeScript:mainfrom RyanCavanaugh:perf-baseline-statsRyanCavanaugh/TypeScript:perf-baseline-statsCopy head branch name to clipboard
RyanCavanaugh merged 10 commits into
microsoft:mainmicrosoft/TypeScript:mainfrom
RyanCavanaugh:perf-baseline-statsRyanCavanaugh/TypeScript:perf-baseline-statsCopy head branch name to clipboard
Conversation
jakebailey
reviewed
Mar 12, 2024
Member
|
What happens if a test uses |
jakebailey
approved these changes
Mar 15, 2024
This was referenced Mar 15, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds automatic baselines similar to what you get from
--extendedDiagnosticswhenever the harness detects an "interesting" amount of something happening. To reduce baseline noise from future changes, these values are rounded to metric-specific intervals, so e.g. if we go from 513 to 517 symbols, that's not going to register a baseline diff (it will print back as "500").This should let us easily create "don't blow stuff up" tests when things like #57710 happen.
I want to make this opt-in/opt-out on a per-test base but the harness code is not well-suited to threading that info through and I want to do a follow-up refactor that cleans up the baseliner to have access to the per-test options.