DotMemoryDiagnoser implementation - #2549
#2549DotMemoryDiagnoser implementation#2549AndreyAkinshin merged 4 commits intodotnet:masterdotnet/BenchmarkDotNet:masterfrom martinothamar:feature/dotMemory-profilermartinothamar/BenchmarkDotNet:feature/dotMemory-profilerCopy head branch name to clipboard
Conversation
|
There was a segfault happening for the integration test that I don't quite understand. I can no longer reproduce it after the last commit. I tried to debug some dumps locally but was unable to find relevant exceptions, so my only guess atm is that the crash originates from unmanaged code. Since changing the init logic seemed to affect things, and the crash always happening during In any case, I would like to test out using only the JetBrains API directly |
|
The last commit simplifies the implementation of the tool by using only the Jetbrains API, and the lifetime of it is more clearly managed based on the incoming signals. I'd be willing to try the same with dotTrace impl. I also want to make a flag like |
|
@martinothamar thank you very much for this PR! |
Updated [BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet) from 0.13.12 to 0.15.8. <details> <summary>Release notes</summary> _Sourced from [BenchmarkDotNet's releases](https://github.com/dotnet/BenchmarkDotNet/releases)._ ## 0.15.8 This release adds OpenMetrics exporter support for Prometheus-compatible metrics export, improves the Roslyn analyzers with multi-target support and better type checking, and fixes several bugs including process deadlocks and WASM trimming issues. ## Features - Add OpenMetrics exporter for Prometheus-compatible metrics output (#2801) - Add Job info to DisassemblyDiagnoser report headers to distinguish assemblies when using multiple coreruns (#2884, fixes #2573) - Add `NO_COLOR` environment variable support for disabling console colors (#2870) ## Improvements - Multi-target analyzers with improved type assignability checking using semantic model (#2866) - Add new analyzer diagnostic BDN1503 for better argument/params validation (#2865, fixes #2864) - Use PolySharp for `[DynamicallyAccessedMembers]` attribute polyfill (#2883) - Refactor to use `AsyncProcessOutputReader` for cleaner process output handling (#2878) ## Bug Fixes - Fix process deadlock issue when reading process output (#2877) - Fix WASM generated project being trimmed out (#2872) - Allow filters to filter out every benchmark from a type without errors (#2879, fixes #2860) - Fix unhandled exception when running `BenchmarkRunner.Run<T>()` with arguments on invalid benchmark type (#2880, fixes #2724) ## Internal - Update release workflow for analyzers (#2882) - Improve docs building workflow - Cleanup `#if`-`#endif` preprocessor directives using PolySharp polyfills (#2881) **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.7...v0.15.8 ## 0.15.7 This release introduces Roslyn analyzers to catch incorrect BenchmarkDotNet usage at compile time, improves .NET Framework version detection, and updates OS detection support. ## Features - Add Roslyn analyzers to detect incorrect usage of BenchmarkDotNet at compile-time (#2837) - Validates benchmark class structure (public, non-sealed, generic constraints) - Checks `[Arguments]`, `[Params]`, and `[ParamsAllValues]` attribute usage - Verifies `[GenericTypeArguments]` requirements - Ensures only one baseline method per category - Validates `BenchmarkRunner.Run` invocations ## Improvements - Improve .NET Framework version detection by retrieving version from `TargetFrameworkAttribute` (#2682) - Bump Perfolizer 0.6.0 → 0.6.1, bringing updated Windows and macOS version detection in `OsBrandHelper` ## Bug Fixes - Fix null reference handling and exception logging in `TestCaseFilter` for the test adapter - Fix flaky CI tests by increasing build timeout values (#2854) ## Internal - Improve release workflow in `release.yaml` **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.6...v0.15.7 ## 0.15.6 ## v0.15.6 This release adds ref struct parameter support for `[ArgumentsSource]`, fixes Native AOT runtime moniker resolution, and upgrades to Perfolizer 0.6.0 with the new Pragmastat statistical engine. ### Features - Add ref struct parameter support for `[ArgumentsSource]` attribute, enabling `Span<T>` and `ReadOnlySpan<char>` parameters (#2849) ### Bug Fixes - Fix runtime moniker normalization for Native AOT targets (#2852) ### Improvements - Upgrade to Perfolizer 0.6.0 with Pragmastat statistical engine integration ### Documentation - Add documentation for breaking changes related to disassembler native dependencies (#2836) ### Internal - Introduce GitHub Actions release workflow **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.5...v0.15.6 ## 0.15.5 This release fixes job naming consistency when using `--runtimes`, clamps histogram bin bounds to avoid confusing negative values, and reduces output directory clutter by filtering unnecessary runtime and satellite assembly files. ## Features - Add custom MSBuild targets to remove unnecessary files from the bin directory (#2737) - Filters out Capstone native binaries for non-target platforms - Removes satellite assemblies from Microsoft.CodeAnalysis packages ## Bug Fixes - Fix job names consistency between `SimpleJobAttribute` and `--runtimes` CLI option (#2841) - Jobs now use runtime names as IDs consistently across all runtime monikers - Clamp histogram bin lower bounds to non-negative values (#1821) - Prevents confusing negative values in histogram output for non-negative measurements ## Internal - Bump Perfolizer: 0.5.3 → 0.5.4 (#2773) - Update changelog and GitHub Pages generation workflows - Enable `workflow_dispatch` for test workflow (#2835) **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.4...v0.15.5 ## 0.15.4 This release fixes issues with `ParamsSource` attribute resolution in inheritance scenarios and corrects a MSBuild syntax error in the TestAdapter. ## Bug Fixes - Allow `[ParamsSource]` to resolve overridden methods and properties in derived classes (#2832) - Fix MSBuild condition syntax for `TestTfmsInParallel` property that prevented Visual Studio from loading projects (#2831) **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.3...v0.15.4 ## 0.15.3 This release brings .NET 10 NativeAOT instruction set support, improved CPU detection on Windows when WMIC is unavailable, test adapter filtering, and numerous bug fixes. ## Breaking Changes - Deprecated `.WithNuget()` job extension in favor of `.WithMsBuildArguments()` (#2812) ## Features - Add VS Test Adapter filter support for running specific benchmarks (#2788) - Update NativeAOT instruction set support for .NET 10+ (#2828) ## Improvements - Add PowerShell-based CPU detection fallback for Windows when WMIC is unavailable (#2749) - Improve `IsNetCore` and `IsNativeAOT` detection for single-file apps without AOT (#2799) - Use `--nodeReuse:false` for dotnet CLI commands to improve build isolation (#2814) - Enable assembly signing for debug builds (#2774) ## Bug Fixes - Fix `ArgumentsSource` on external types not working if the argument type is not primitive (#2820) - Fix workload warmup mode not working correctly - Fix `EtwProfiler` for file paths slightly under 260 characters (#2808) - Fix console logs being output twice when using TestAdapter (#2790) - Fix `EventProcessor.OnEndValidationStage` not being called when critical validation errors occur (#2816) - Fix `XmlException` thrown when `TextReader.Null` is passed to `AppConfigGenerator` (#2817) - Fix case sensitivity issue in `NativeMemoryLogParser` program name matching (#2795) - Fix typo in `BuildPlots.R` ## Internal - Replace StyleCop.Analyzers with unstable version for improved analysis (#2796) - Add workflow to run selected tests (#2797) - Fix flaky `MemoryDiagnoser` tests on macOS (#2813) - Fix x86 disassembler tests for net462 (#2792) - Split `TimeConsumingBenchmark` class to reduce test time - Update `BenchmarkDotNetDiagnosers` package version (#2805) - Fix comment in package props about `GenerateProgramFile` (#2802) **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.2...v0.15.3 ## 0.15.2 This release improves memory allocation measurement accuracy and adds new features for job ordering and runtime validation. ## Features - Add `JobOrderPolicy` option to sort jobs in numeric order instead of ordinal order (#2770) - Add `RuntimeValidator` to detect benchmarks with null runtime configuration (#2771) ## Improvements - Improve memory diagnoser accuracy with better allocation measurement isolation, warm-up phase, and handling of tiered JIT (#2562) ## Bug Fixes - Auto-generate unique job IDs between benchmark runs to prevent ID collisions - Skip null runtime validation for in-process toolchain (#2780) - Fix flaky memory allocation test (#2782) - Fix benchmark test adapter enumeration issues (#2766) ## Internal - Modify macOS runner image for CI (#2775) - Add setting to skip test reports when original workflow is cancelled (#2772) - Suppress xunit non-serializable data warnings (#2769) - Enable `--force-clone` for docs-fetch in generate-gh-pages workflow - Allow `workflow_dispatch` for publish-nightly workflow - Enhance docs-fetch command with additional options - Remove `docs/_changelog` folder from main branch (migrated to docs-changelog branch) **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.1...v0.15.2 ## 0.15.1 A maintenance release with improved cross-platform compatibility, a new feature for referencing external types in source attributes, and several bug fixes for ARM CPUs and unsupported operating systems. ## Features - Allow `[ArgumentsSource]` and `[ParamsSource]` to reference methods in other types via new constructor overload: `[ArgumentsSource(typeof(MyClass), nameof(MyClass.Values))]` (#2748) ## Bug Fixes - Fix WakeLock P/Invoke compatibility with ARM CPUs by refactoring `REASON_CONTEXT` to use proper union structure (#2745, #2756) - Fix `Console.CancelKeyPress` crash on platforms that don't support it (Android, iOS, tvOS, WASM) (#2739, #2741) - Fix CPU detection crash on unsupported operating systems by returning `CpuInfo.Unknown` (#2740) - Support `.slnx` solution file format when searching for solution files (#2764) ## Improvements - Bump Perfolizer: 0.5.2 → 0.5.3 - Make `ExporterBase.GetArtifactFullName` accessibility modifier more permissive ## Internal - Update .NET SDK version to 8.0.410 (#2762) - Update Microsoft.NET.Test.Sdk and other package dependencies (#2750, #2755) - Rework changelog generation to use `docs-changelog` branch (#93d12c42) - Fix line-endings to LF in several files - Update GitHub Actions workflows **Full Changelog**: dotnet/BenchmarkDotNet@v0.15.0...v0.15.1 ## 0.15.0 BenchmarkDotNet v0.15.0 brings .NET 10 support, a new WakeLock feature to prevent system sleep during benchmarks, improved engine internals for more consistent measurements, and numerous bug fixes and improvements. ## Features - **WakeLock support**: New `[WakeLock]` attribute and `--wakeLock` CLI option to prevent the system from entering sleep mode while benchmarks are running (#2670) - **.NET 10 support**: Added `RuntimeMoniker.Net10`, `NativeAot10`, and `Mono10` with full toolchain support (#2642) - **Box plots in ScottPlotExporter**: New box plot visualization for benchmark results with improved font sizing - **RiscV64 platform support**: Added `Platform.RiscV64` for RISC-V 64-bit architecture (#2644, #2647) - **Required properties support**: Benchmark classes can now use C# 11 `required` properties (#2579) - **HostSignal.AfterProcessStart**: New signal allows diagnosers to obtain the process ID of a benchmark process started in suspended state (#2674) - **Parallel build control**: New `ConfigOptions.DisableParallelBuild` option to force sequential builds (#2725) - **Auto-hide empty metric columns**: `ThreadingDiagnoser` and `ExceptionDiagnoser` now support configuration to hide columns when metrics have no values (#2673) - **Measurements in DiagnoserResults**: Custom diagnosers can now access measurements for calculations (#2731) ## Improvements - **Constant stack size engine refactoring**: Engine stages refactored to use `IEngineStageEvaluator` for more consistent instruction location and simpler code (#2688) - **Use ArtifactsPath instead of IntermediateOutputPath**: Improved build artifact handling for SDK 8+ (#2676) - **InProcessNoEmitRunner NativeAOT support**: Basic support for running `InProcessNoEmitRunner` with NativeAOT (#2702) - **Allow ParamsAttribute values from derived classes**: `ParamsAttribute.Values` setter is now protected instead of private (#2716) - **Updated clrmd to 3.1**: Disassembler now uses `ClrMdV3Disassembler` (#2488) - **Updated ScottPlot to 5.0.54**: Plotting exporter updated to latest ScottPlot version (#2709) - **Perfolizer upgraded to 0.5.2**: CPU/OS detection logic moved to Perfolizer with new `Perfonar` exporters replacing `Phd` exporters - **Log warnings for empty benchmarks**: Clear warning messages when running benchmarks that match no methods (#2718) - **Validation for sealed benchmark classes**: Compiler now warns when benchmark classes are sealed (#2660) - **Improved baseline warning message**: More descriptive warning when baseline benchmarks are misconfigured (#2650) ## Bug Fixes - **Fix async GlobalSetup/GlobalCleanup with InProcessEmit**: Async setup and cleanup methods are now properly awaited (#2109) - **Fix Windows path too long**: Handle Windows MAX_PATH limitations in build paths (#2681) - **Fix builds with --keepFiles**: Include auto-incremented ID in build artifacts directory to avoid conflicts (#2423) - **Fix Ctrl-C handling**: System state (power management, console title) is now properly reverted on process termination (#2483, #2661) - **Fix dotnet command failure detection**: Commands now always fail when dotnet returns non-zero exit code (#2535) - **Fix deadlock in GetDotNetSdkVersion**: Resolved potential deadlock in SDK version detection (#2622) - **Fix lscpu CPU frequency parsing**: Corrected frequency parsing from lscpu output - **Make lscpu call language-invariant**: CPU detection now works correctly regardless of system locale (#2577) - **Clean up unsupported Native AOT flags**: Removed obsolete `IlcGenerateCompleteTypeMetadata` and updated flag names (#2616) - **Native AOT projects copy SettingsWeWantToCopy**: Build settings are now properly propagated (#2665) ## Breaking Changes - **Removed ConfigCompatibilityValidator**: No longer validates config compatibility between runs (#2599) - **Phd exporters renamed to Perfonar**: `[PhdExporter]` → `[PerfonarExporter]`, `PhdJsonExporter` → `PerfonarJsonExporter`, `PhdMdExporter` → `PerfonarMdExporter` - **Removed netstandard1.0 target from Annotations package**: Minimum target is now netstandard2.0 ## Documentation - Added Visual Studio Profiler documentation with samples (#2672) - Added WakeLock documentation and samples (#2670) ... (truncated) ## 0.14.0 Full changelog: https://benchmarkdotnet.org/changelog/v0.14.0.html ## Highlights * Introduce `BenchmarkDotNet.Diagnostics.dotMemory` [#2549](dotnet/BenchmarkDotNet#2549): memory allocation profile of your benchmarks using [dotMemory](https://www.jetbrains.com/dotmemory/), see @BenchmarkDotNet.Samples.IntroDotMemoryDiagnoser * Introduce `BenchmarkDotNet.Exporters.Plotting` [#2560](dotnet/BenchmarkDotNet#2560): plotting via [ScottPlot](https://scottplot.net/) (initial version) * Multiple bugfixes * The default build toolchains have been updated to pass `IntermediateOutputPath`, `OutputPath`, and `OutDir` properties to the `dotnet build` command. This change forces all build outputs to be placed in a new directory generated by BenchmarkDotNet, and fixes many issues that have been reported with builds. You can also access these paths in your own `.csproj` and `.props` from those properties if you need to copy custom files to the output. ## Bug fixes * Fixed multiple build-related bugs including passing MsBuildArguments and .Net 8's `UseArtifactsOutput`. ## Breaking Changes * `DotNetCliBuilder` removed `retryFailedBuildWithNoDeps` constructor option. * `DotNetCliCommand` removed `RetryFailedBuildWithNoDeps` property and `BuildNoRestoreNoDependencies()` and `PublishNoBuildAndNoRestore()` methods (replaced with `PublishNoRestore()`). Commits viewable in [compare view](dotnet/BenchmarkDotNet@v0.13.12...v0.15.8). </details> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Implements
DotMemoryDiagnoserto matchDotTraceDiagnoserCommonpackage or just use<Compile Include="..." />Sample output of
IntroDotMemoryDiagnoser