Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

DotMemoryDiagnoser implementation - #2549

#2549
Merged
AndreyAkinshin merged 4 commits into
dotnet:masterdotnet/BenchmarkDotNet:masterfrom
martinothamar:feature/dotMemory-profilermartinothamar/BenchmarkDotNet:feature/dotMemory-profilerCopy head branch name to clipboard
Apr 1, 2024
Merged

DotMemoryDiagnoser implementation#2549
AndreyAkinshin merged 4 commits into
dotnet:masterdotnet/BenchmarkDotNet:masterfrom
martinothamar:feature/dotMemory-profilermartinothamar/BenchmarkDotNet:feature/dotMemory-profilerCopy head branch name to clipboard

Conversation

@martinothamar

Copy link
Copy Markdown
Contributor

Implements DotMemoryDiagnoser to match DotTraceDiagnoser

  • The code is pretty similar between dotMemory and dotTrace impl, because the underlying APIs/CLIs are similar. Could do something to share code here
    • Is there any reason we can't use the PID config on the APIs for the Jetbrains library directly? I could refactor to use that instead, both for dotTrace and dotMemory
  • Could refactor out common code and either publish a Common package or just use <Compile Include="..." />

Sample output of IntroDotMemoryDiagnoser

image

@martinothamar

Copy link
Copy Markdown
Contributor Author

There was a segfault happening for the integration test that I don't quite understand. I can no longer reproduce it after the last commit. I tried to debug some dumps locally but was unable to find relevant exceptions, so my only guess atm is that the crash originates from unmanaged code. Since changing the init logic seemed to affect things, and the crash always happening during InProcessEmitToolchain, I'm guessing it has something to do with attaching/detaching multiple times in the same process or something

In any case, I would like to test out using only the JetBrains API directly

@martinothamar

martinothamar commented Mar 31, 2024

Copy link
Copy Markdown
Contributor Author

The last commit simplifies the implementation of the tool by using only the Jetbrains API, and the lifetime of it is more clearly managed based on the incoming signals.

I'd be willing to try the same with dotTrace impl. I also want to make a flag like performExtraBenchmarksRun similar to some of the other profilers. I could do these en subsequent PRs

@AndreyAkinshin
AndreyAkinshin merged commit c7b7abf into dotnet:master Apr 1, 2024
@AndreyAkinshin AndreyAkinshin added this to the v0.14.0 milestone Apr 1, 2024
@AndreyAkinshin

Copy link
Copy Markdown
Member

@martinothamar thank you very much for this PR!

@martinothamar
martinothamar deleted the feature/dotMemory-profiler branch April 1, 2024 15:42
liuhaoyang pushed a commit to dotnetcore/mocha that referenced this pull request Jul 27, 2026
Updated [BenchmarkDotNet](https://github.com/dotnet/BenchmarkDotNet)
from 0.13.12 to 0.15.8.

<details>
<summary>Release notes</summary>

_Sourced from [BenchmarkDotNet's
releases](https://github.com/dotnet/BenchmarkDotNet/releases)._

## 0.15.8

This release adds OpenMetrics exporter support for Prometheus-compatible
metrics export, improves the Roslyn analyzers with multi-target support
and better type checking, and fixes several bugs including process
deadlocks and WASM trimming issues.

## Features

- Add OpenMetrics exporter for Prometheus-compatible metrics output
(#​2801)
- Add Job info to DisassemblyDiagnoser report headers to distinguish
assemblies when using multiple coreruns (#​2884, fixes #​2573)
- Add `NO_COLOR` environment variable support for disabling console
colors (#​2870)

## Improvements

- Multi-target analyzers with improved type assignability checking using
semantic model (#​2866)
- Add new analyzer diagnostic BDN1503 for better argument/params
validation (#​2865, fixes #​2864)
- Use PolySharp for `[DynamicallyAccessedMembers]` attribute polyfill
(#​2883)
- Refactor to use `AsyncProcessOutputReader` for cleaner process output
handling (#​2878)

## Bug Fixes

- Fix process deadlock issue when reading process output (#​2877)
- Fix WASM generated project being trimmed out (#​2872)
- Allow filters to filter out every benchmark from a type without errors
(#​2879, fixes #​2860)
- Fix unhandled exception when running `BenchmarkRunner.Run<T>()` with
arguments on invalid benchmark type (#​2880, fixes #​2724)

## Internal

- Update release workflow for analyzers (#​2882)
- Improve docs building workflow
- Cleanup `#if`-`#endif` preprocessor directives using PolySharp
polyfills (#​2881)

**Full Changelog**:
dotnet/BenchmarkDotNet@v0.15.7...v0.15.8


## 0.15.7

This release introduces Roslyn analyzers to catch incorrect
BenchmarkDotNet usage at compile time, improves .NET Framework version
detection, and updates OS detection support.

## Features

- Add Roslyn analyzers to detect incorrect usage of BenchmarkDotNet at
compile-time (#​2837)
- Validates benchmark class structure (public, non-sealed, generic
constraints)
- Checks `[Arguments]`, `[Params]`, and `[ParamsAllValues]` attribute
usage
  - Verifies `[GenericTypeArguments]` requirements
  - Ensures only one baseline method per category
  - Validates `BenchmarkRunner.Run` invocations

## Improvements

- Improve .NET Framework version detection by retrieving version from
`TargetFrameworkAttribute` (#​2682)
- Bump Perfolizer 0.6.0 → 0.6.1, bringing updated Windows and macOS
version detection in `OsBrandHelper`

## Bug Fixes

- Fix null reference handling and exception logging in `TestCaseFilter`
for the test adapter
- Fix flaky CI tests by increasing build timeout values (#​2854)

## Internal

- Improve release workflow in `release.yaml`

**Full Changelog**:
dotnet/BenchmarkDotNet@v0.15.6...v0.15.7


## 0.15.6

## v0.15.6

This release adds ref struct parameter support for `[ArgumentsSource]`,
fixes Native AOT runtime moniker resolution, and upgrades to Perfolizer
0.6.0 with the new Pragmastat statistical engine.

### Features

- Add ref struct parameter support for `[ArgumentsSource]` attribute,
enabling `Span<T>` and `ReadOnlySpan<char>` parameters (#​2849)

### Bug Fixes

- Fix runtime moniker normalization for Native AOT targets (#​2852)

### Improvements

- Upgrade to Perfolizer 0.6.0 with Pragmastat statistical engine
integration

### Documentation

- Add documentation for breaking changes related to disassembler native
dependencies (#​2836)

### Internal

- Introduce GitHub Actions release workflow

**Full Changelog**:
dotnet/BenchmarkDotNet@v0.15.5...v0.15.6


## 0.15.5

This release fixes job naming consistency when using `--runtimes`,
clamps histogram bin bounds to avoid confusing negative values, and
reduces output directory clutter by filtering unnecessary runtime and
satellite assembly files.

## Features

- Add custom MSBuild targets to remove unnecessary files from the bin
directory (#​2737)
  - Filters out Capstone native binaries for non-target platforms
  - Removes satellite assemblies from Microsoft.CodeAnalysis packages

## Bug Fixes

- Fix job names consistency between `SimpleJobAttribute` and
`--runtimes` CLI option (#​2841)
- Jobs now use runtime names as IDs consistently across all runtime
monikers
- Clamp histogram bin lower bounds to non-negative values (#​1821)
- Prevents confusing negative values in histogram output for
non-negative measurements

## Internal

- Bump Perfolizer: 0.5.3 → 0.5.4 (#​2773)
- Update changelog and GitHub Pages generation workflows
- Enable `workflow_dispatch` for test workflow (#​2835)

**Full Changelog**:
dotnet/BenchmarkDotNet@v0.15.4...v0.15.5


## 0.15.4

This release fixes issues with `ParamsSource` attribute resolution in
inheritance scenarios and corrects a MSBuild syntax error in the
TestAdapter.

## Bug Fixes

- Allow `[ParamsSource]` to resolve overridden methods and properties in
derived classes (#​2832)
- Fix MSBuild condition syntax for `TestTfmsInParallel` property that
prevented Visual Studio from loading projects (#​2831)

**Full Changelog**:
dotnet/BenchmarkDotNet@v0.15.3...v0.15.4


## 0.15.3

This release brings .NET 10 NativeAOT instruction set support, improved
CPU detection on Windows when WMIC is unavailable, test adapter
filtering, and numerous bug fixes.

## Breaking Changes

- Deprecated `.WithNuget()` job extension in favor of
`.WithMsBuildArguments()` (#​2812)

## Features

- Add VS Test Adapter filter support for running specific benchmarks
(#​2788)
- Update NativeAOT instruction set support for .NET 10+ (#​2828)

## Improvements

- Add PowerShell-based CPU detection fallback for Windows when WMIC is
unavailable (#​2749)
- Improve `IsNetCore` and `IsNativeAOT` detection for single-file apps
without AOT (#​2799)
- Use `--nodeReuse:false` for dotnet CLI commands to improve build
isolation (#​2814)
- Enable assembly signing for debug builds (#​2774)

## Bug Fixes

- Fix `ArgumentsSource` on external types not working if the argument
type is not primitive (#​2820)
- Fix workload warmup mode not working correctly
- Fix `EtwProfiler` for file paths slightly under 260 characters
(#​2808)
- Fix console logs being output twice when using TestAdapter (#​2790)
- Fix `EventProcessor.OnEndValidationStage` not being called when
critical validation errors occur (#​2816)
- Fix `XmlException` thrown when `TextReader.Null` is passed to
`AppConfigGenerator` (#​2817)
- Fix case sensitivity issue in `NativeMemoryLogParser` program name
matching (#​2795)
- Fix typo in `BuildPlots.R`

## Internal

- Replace StyleCop.Analyzers with unstable version for improved analysis
(#​2796)
- Add workflow to run selected tests (#​2797)
- Fix flaky `MemoryDiagnoser` tests on macOS (#​2813)
- Fix x86 disassembler tests for net462 (#​2792)
- Split `TimeConsumingBenchmark` class to reduce test time
- Update `BenchmarkDotNetDiagnosers` package version (#​2805)
- Fix comment in package props about `GenerateProgramFile` (#​2802)

**Full Changelog**:
dotnet/BenchmarkDotNet@v0.15.2...v0.15.3


## 0.15.2

This release improves memory allocation measurement accuracy and adds
new features for job ordering and runtime validation.

## Features

- Add `JobOrderPolicy` option to sort jobs in numeric order instead of
ordinal order (#​2770)
- Add `RuntimeValidator` to detect benchmarks with null runtime
configuration (#​2771)

## Improvements

- Improve memory diagnoser accuracy with better allocation measurement
isolation, warm-up phase, and handling of tiered JIT (#​2562)

## Bug Fixes

- Auto-generate unique job IDs between benchmark runs to prevent ID
collisions
- Skip null runtime validation for in-process toolchain (#​2780)
- Fix flaky memory allocation test (#​2782)
- Fix benchmark test adapter enumeration issues (#​2766)

## Internal

- Modify macOS runner image for CI (#​2775)
- Add setting to skip test reports when original workflow is cancelled
(#​2772)
- Suppress xunit non-serializable data warnings (#​2769)
- Enable `--force-clone` for docs-fetch in generate-gh-pages workflow
- Allow `workflow_dispatch` for publish-nightly workflow
- Enhance docs-fetch command with additional options
- Remove `docs/_changelog` folder from main branch (migrated to
docs-changelog branch)

**Full Changelog**:
dotnet/BenchmarkDotNet@v0.15.1...v0.15.2


## 0.15.1

A maintenance release with improved cross-platform compatibility, a new
feature for referencing external types in source attributes, and several
bug fixes for ARM CPUs and unsupported operating systems.

## Features

- Allow `[ArgumentsSource]` and `[ParamsSource]` to reference methods in
other types via new constructor overload:
`[ArgumentsSource(typeof(MyClass), nameof(MyClass.Values))]` (#​2748)

## Bug Fixes

- Fix WakeLock P/Invoke compatibility with ARM CPUs by refactoring
`REASON_CONTEXT` to use proper union structure (#​2745, #​2756)
- Fix `Console.CancelKeyPress` crash on platforms that don't support it
(Android, iOS, tvOS, WASM) (#​2739, #​2741)
- Fix CPU detection crash on unsupported operating systems by returning
`CpuInfo.Unknown` (#​2740)
- Support `.slnx` solution file format when searching for solution files
(#​2764)

## Improvements

- Bump Perfolizer: 0.5.2 → 0.5.3
- Make `ExporterBase.GetArtifactFullName` accessibility modifier more
permissive

## Internal

- Update .NET SDK version to 8.0.410 (#​2762)
- Update Microsoft.NET.Test.Sdk and other package dependencies (#​2750,
#​2755)
- Rework changelog generation to use `docs-changelog` branch
(#​93d12c42)
- Fix line-endings to LF in several files
- Update GitHub Actions workflows

**Full Changelog**:
dotnet/BenchmarkDotNet@v0.15.0...v0.15.1


## 0.15.0

BenchmarkDotNet v0.15.0 brings .NET 10 support, a new WakeLock feature
to prevent system sleep during benchmarks, improved engine internals for
more consistent measurements, and numerous bug fixes and improvements.

## Features

- **WakeLock support**: New `[WakeLock]` attribute and `--wakeLock` CLI
option to prevent the system from entering sleep mode while benchmarks
are running (#​2670)
- **.NET 10 support**: Added `RuntimeMoniker.Net10`, `NativeAot10`, and
`Mono10` with full toolchain support (#​2642)
- **Box plots in ScottPlotExporter**: New box plot visualization for
benchmark results with improved font sizing
- **RiscV64 platform support**: Added `Platform.RiscV64` for RISC-V
64-bit architecture (#​2644, #​2647)
- **Required properties support**: Benchmark classes can now use C# 11
`required` properties (#​2579)
- **HostSignal.AfterProcessStart**: New signal allows diagnosers to
obtain the process ID of a benchmark process started in suspended state
(#​2674)
- **Parallel build control**: New `ConfigOptions.DisableParallelBuild`
option to force sequential builds (#​2725)
- **Auto-hide empty metric columns**: `ThreadingDiagnoser` and
`ExceptionDiagnoser` now support configuration to hide columns when
metrics have no values (#​2673)
- **Measurements in DiagnoserResults**: Custom diagnosers can now access
measurements for calculations (#​2731)

## Improvements

- **Constant stack size engine refactoring**: Engine stages refactored
to use `IEngineStageEvaluator` for more consistent instruction location
and simpler code (#​2688)
- **Use ArtifactsPath instead of IntermediateOutputPath**: Improved
build artifact handling for SDK 8+ (#​2676)
- **InProcessNoEmitRunner NativeAOT support**: Basic support for running
`InProcessNoEmitRunner` with NativeAOT (#​2702)
- **Allow ParamsAttribute values from derived classes**:
`ParamsAttribute.Values` setter is now protected instead of private
(#​2716)
- **Updated clrmd to 3.1**: Disassembler now uses `ClrMdV3Disassembler`
(#​2488)
- **Updated ScottPlot to 5.0.54**: Plotting exporter updated to latest
ScottPlot version (#​2709)
- **Perfolizer upgraded to 0.5.2**: CPU/OS detection logic moved to
Perfolizer with new `Perfonar` exporters replacing `Phd` exporters
- **Log warnings for empty benchmarks**: Clear warning messages when
running benchmarks that match no methods (#​2718)
- **Validation for sealed benchmark classes**: Compiler now warns when
benchmark classes are sealed (#​2660)
- **Improved baseline warning message**: More descriptive warning when
baseline benchmarks are misconfigured (#​2650)

## Bug Fixes

- **Fix async GlobalSetup/GlobalCleanup with InProcessEmit**: Async
setup and cleanup methods are now properly awaited (#​2109)
- **Fix Windows path too long**: Handle Windows MAX_PATH limitations in
build paths (#​2681)
- **Fix builds with --keepFiles**: Include auto-incremented ID in build
artifacts directory to avoid conflicts (#​2423)
- **Fix Ctrl-C handling**: System state (power management, console
title) is now properly reverted on process termination (#​2483, #​2661)
- **Fix dotnet command failure detection**: Commands now always fail
when dotnet returns non-zero exit code (#​2535)
- **Fix deadlock in GetDotNetSdkVersion**: Resolved potential deadlock
in SDK version detection (#​2622)
- **Fix lscpu CPU frequency parsing**: Corrected frequency parsing from
lscpu output
- **Make lscpu call language-invariant**: CPU detection now works
correctly regardless of system locale (#​2577)
- **Clean up unsupported Native AOT flags**: Removed obsolete
`IlcGenerateCompleteTypeMetadata` and updated flag names (#​2616)
- **Native AOT projects copy SettingsWeWantToCopy**: Build settings are
now properly propagated (#​2665)

## Breaking Changes

- **Removed ConfigCompatibilityValidator**: No longer validates config
compatibility between runs (#​2599)
- **Phd exporters renamed to Perfonar**: `[PhdExporter]` →
`[PerfonarExporter]`, `PhdJsonExporter` → `PerfonarJsonExporter`,
`PhdMdExporter` → `PerfonarMdExporter`
- **Removed netstandard1.0 target from Annotations package**: Minimum
target is now netstandard2.0

## Documentation

- Added Visual Studio Profiler documentation with samples (#​2672)
- Added WakeLock documentation and samples (#​2670)
 ... (truncated)

## 0.14.0

Full changelog: https://benchmarkdotnet.org/changelog/v0.14.0.html

## Highlights

* Introduce `BenchmarkDotNet.Diagnostics.dotMemory`
[#​2549](dotnet/BenchmarkDotNet#2549): memory
allocation profile of your benchmarks using
[dotMemory](https://www.jetbrains.com/dotmemory/), see
@​BenchmarkDotNet.Samples.IntroDotMemoryDiagnoser
* Introduce `BenchmarkDotNet.Exporters.Plotting`
[#​2560](dotnet/BenchmarkDotNet#2560): plotting
via [ScottPlot](https://scottplot.net/) (initial version)
* Multiple bugfixes
* The default build toolchains have been updated to pass
`IntermediateOutputPath`, `OutputPath`, and `OutDir` properties to the
`dotnet build` command. This change forces all build outputs to be
placed in a new directory generated by BenchmarkDotNet, and fixes many
issues that have been reported with builds. You can also access these
paths in your own `.csproj` and `.props` from those properties if you
need to copy custom files to the output.

## Bug fixes

* Fixed multiple build-related bugs including passing MsBuildArguments
and .Net 8's `UseArtifactsOutput`.

## Breaking Changes

* `DotNetCliBuilder` removed `retryFailedBuildWithNoDeps` constructor
option.
* `DotNetCliCommand` removed `RetryFailedBuildWithNoDeps` property and
`BuildNoRestoreNoDependencies()` and `PublishNoBuildAndNoRestore()`
methods (replaced with `PublishNoRestore()`).

Commits viewable in [compare
view](dotnet/BenchmarkDotNet@v0.13.12...v0.15.8).
</details>

[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=BenchmarkDotNet&package-manager=nuget&previous-version=0.13.12&new-version=0.15.8)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.