Remove slow regex from threading analyzers - #1547
#1547Remove slow regex from threading analyzers#1547AArnott merged 3 commits intomainmicrosoft/vs-threading:mainfrom dev/andarno/fixregextimemicrosoft/vs-threading:dev/andarno/fixregextimeCopy head branch name to clipboard
Conversation
There was a problem hiding this comment.
Pull request overview
Removes regex-based parsing from threading analyzers to avoid pathological backtracking/timeout scenarios and improve performance determinism.
Changes:
- Removed regex fields (and timeout handling) used to parse additional-file entries.
- Added non-regex parsers for type/member reference lines and updated call sites to use them.
a56023c to
13513b5
Compare
13513b5 to
4abe4eb
Compare
4abe4eb to
0df6788
Compare
0df6788 to
c4644e8
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
src/Microsoft.VisualStudio.Threading.Analyzers/CommonInterest.cs:91
SplitQualifiedIdentifierreturns a non-nullstringtype name, but the deconstruction usesstring? typeName, which can trigger nullable warnings when passingtypeNametoQualifiedType(expectsstring). Consider deconstructing intostring typeName(or otherwise ensure the value is statically non-null).
(ImmutableArray<string> containingNamespace, string? typeName) = SplitQualifiedIdentifier(typeNameMemory);
var type = new QualifiedType(containingNamespace, typeName);
QualifiedMember member = memberNameValue is not null ? new QualifiedMember(type, memberNameValue) : default(QualifiedMember);
| } | ||
| catch (RegexMatchTimeoutException) | ||
|
|
||
| (ImmutableArray<string> containingNamespace, string? typeName) = SplitQualifiedIdentifier(typeNameMemory); |
There was a problem hiding this comment.
Same nullable flow issue here: SplitQualifiedIdentifier returns a non-null string, but you deconstruct into string? typeName and then pass it to QualifiedType, which may produce nullable warnings. Use a non-null string local (or apply a null-forgiving operator if truly intended).
This issue also appears on line 89 of the same file.
| (ImmutableArray<string> containingNamespace, string? typeName) = SplitQualifiedIdentifier(typeNameMemory); | |
| (ImmutableArray<string> containingNamespace, string typeName) = SplitQualifiedIdentifier(typeNameMemory); |
| // Copyright (c) Microsoft Corporation. All rights reserved. | ||
| // Licensed under the MIT license. See LICENSE file in the project root for full license information. | ||
|
|
||
| using Microsoft.VisualStudio.Threading.Analyzers; |
There was a problem hiding this comment.
using Microsoft.VisualStudio.Threading.Analyzers; is redundant here because the test project already defines global using Microsoft.VisualStudio.Threading.Analyzers; in Usings.cs. Keeping both can cause unnecessary-using style diagnostics in some configurations.
| using Microsoft.VisualStudio.Threading.Analyzers; |
Updated [Microsoft.Bcl.AsyncInterfaces](https://github.com/dotnet/dotnet) from 10.0.9 to 10.0.10. <details> <summary>Release notes</summary> _Sourced from [Microsoft.Bcl.AsyncInterfaces's releases](https://github.com/dotnet/dotnet/releases)._ No release notes found for this version range. Commits viewable in [compare view](https://github.com/dotnet/dotnet/commits). </details> Updated [Microsoft.Bcl.Memory](https://github.com/dotnet/dotnet) from 10.0.9 to 10.0.10. <details> <summary>Release notes</summary> _Sourced from [Microsoft.Bcl.Memory's releases](https://github.com/dotnet/dotnet/releases)._ No release notes found for this version range. Commits viewable in [compare view](https://github.com/dotnet/dotnet/commits). </details> Updated [Microsoft.Bcl.TimeProvider](https://github.com/dotnet/dotnet) from 10.0.9 to 10.0.10. <details> <summary>Release notes</summary> _Sourced from [Microsoft.Bcl.TimeProvider's releases](https://github.com/dotnet/dotnet/releases)._ No release notes found for this version range. Commits viewable in [compare view](https://github.com/dotnet/dotnet/commits). </details> Updated [Microsoft.CodeAnalysis.Analyzers](https://github.com/dotnet/roslyn) from 5.3.0 to 5.6.0. <details> <summary>Release notes</summary> _Sourced from [Microsoft.CodeAnalysis.Analyzers's releases](https://github.com/dotnet/roslyn/releases)._ No release notes found for this version range. Commits viewable in [compare view](https://github.com/dotnet/roslyn/commits). </details> Updated Microsoft.CodeAnalysis.CSharp from 5.3.0 to 5.6.0. Updated Microsoft.CodeAnalysis.CSharp.Workspaces from 5.3.0 to 5.6.0. Updated [Microsoft.Extensions.ObjectPool](https://github.com/dotnet/dotnet) from 10.0.9 to 10.0.10. <details> <summary>Release notes</summary> _Sourced from [Microsoft.Extensions.ObjectPool's releases](https://github.com/dotnet/dotnet/releases)._ No release notes found for this version range. Commits viewable in [compare view](https://github.com/dotnet/dotnet/commits). </details> Updated [Microsoft.NET.Test.Sdk](https://github.com/microsoft/vstest) from 18.5.1 to 18.8.1. <details> <summary>Release notes</summary> _Sourced from [Microsoft.NET.Test.Sdk's releases](https://github.com/microsoft/vstest/releases)._ ## 18.8.1 ## What's Changed * Fix protocol negotiation timeout when STJ reflection is disabled (18.8.1) by @nohwnd in microsoft/vstest#16281 **Full Changelog**: microsoft/vstest@v18.8.0...v18.8.1 ## 18.8.0 ## What's Changed * Migrate from Newtonsoft.Json to System.Text.Json / Jsonite (merge to main) by @nohwnd in microsoft/vstest#15687 - For more detail refer to https://devblogs.microsoft.com/dotnet/vs-test-is-removing-its-newtonsoft-json-dependency/ * Create source-only filter package by @Youssef1313 in microsoft/vstest#15638 * Add ARM64 msdia140.dll support to test platform packages by @nohwnd in microsoft/vstest#15692 * Fix mutex cleanup crash on macOS/Linux by @nohwnd in microsoft/vstest#15684 * Restrict artifact temp directory permissions on Unix by @nohwnd in microsoft/vstest#15729 * Add support for filtering uncategorized tests with TestCategory=None by @Evangelink in microsoft/vstest#15727 * Fix SCI binding failure in DTA hosts (main) by @nohwnd in microsoft/vstest#15724 * Fix HTML logger parallel file collision by @nohwnd in microsoft/vstest#15435 * Improve error message when testhost cannot be found by @nohwnd in microsoft/vstest#16053 * Fix HTML logger exception on invalid XML chars in test display names by @nohwnd in microsoft/vstest#16051 **Full Changelog**: microsoft/vstest@v18.7.0...v18.8.0 ## 18.7.0 ## What's Changed * Add ARM64 msdia140.dll support to test platform packages by @jamesmcroft in microsoft/vstest#15689 * Update System.Memory from 4.5.5 to 4.6.3 by @nohwnd in microsoft/vstest#15706 ## New Contributors * @jamesmcroft made their first contribution in microsoft/vstest#15689 **Full Changelog**: microsoft/vstest@v18.6.0...v18.7.0 ## 18.6.0 ## What's Changed * Revert removal of Video Recorder by @nohwnd in microsoft/vstest#15336 * Speed up blame by filtering non-.NET processes from dump collection by @nohwnd in microsoft/vstest#15518 * Add README.md to NuGet packages by @nohwnd in microsoft/vstest#15550 * Report child process info on connection timeout by @nohwnd in microsoft/vstest#15603 ### Changes to tests and infra * Brand as 18.6 by @nohwnd in microsoft/vstest#15423 * Upgrading code coverage version to 18.5.1, by @fhnaseer in microsoft/vstest#15422 * Updating System.Collections.Immutable to 9.0.11 by @MSLukeWest in microsoft/vstest#15425 * Fix attachVS when used for debugging integration tests by @nohwnd in microsoft/vstest#15451 * Replace dotnet.config, with global.json by @nohwnd in microsoft/vstest#15449 * Document debugging integration tests with AttachVS by @Copilot in microsoft/vstest#15452 * Fix stack overflow tests by @nohwnd in microsoft/vstest#15461 * Make TestAssets.sln buildable locally by @Youssef1313 in microsoft/vstest#15466 * Try filtering out tests by @nohwnd in microsoft/vstest#15463 * Build just once when tfms run in parallel by @nohwnd in microsoft/vstest#15465 * Review simplify compatibility sources, deduplicate tests by @nohwnd in microsoft/vstest#15472 * Cleanup dead TRX code by @Youssef1313 in microsoft/vstest#15474 * Update .NET runtimes to 8.0.25, 9.0.14, and 10.0.4 by @nohwnd in microsoft/vstest#15481 * Compat matrix checker by @nohwnd in microsoft/vstest#15480 * Add trx analysis skill by @nohwnd in microsoft/vstest#15486 * Split integration tests to single tfm and multi tfm project by @nohwnd in microsoft/vstest#15484 * Update matrix by @nohwnd in microsoft/vstest#15477 * Break infinite restore loop in VS by @nohwnd in microsoft/vstest#15503 * Use global package cache for build, and local for running integration tests by @nohwnd in microsoft/vstest#15500 * Update contributing by @nohwnd in microsoft/vstest#15505 * Reduce test wall-clock time by increasing minThreads by @drognanar in microsoft/vstest#15502 * Indicator flakiness by @nohwnd in microsoft/vstest#15513 * Fix ci build by @nohwnd in microsoft/vstest#15515 * Fix thread safety issues by @Evangelink in microsoft/vstest#15512 * Optimize DotnetSDKSimulation_PostProcessing test (163s → 61s) by @nohwnd in microsoft/vstest#15516 * Build isolated test assets for single TFM instead of 7 by @nohwnd in microsoft/vstest#15517 * Remove unused dependencies from Library.IntegrationTests by @nohwnd in microsoft/vstest#15527 * Remove printing _attachments content to console by @nohwnd in microsoft/vstest#15520 * Add Linux/macOS test filtering guide to CONTRIBUTING.md by @nohwnd in microsoft/vstest#15521 * Change integration test parallelization from ClassLevel to MethodLevel by @nohwnd in microsoft/vstest#15526 * Unify target framework checks with IsNetFrameworkTarget/IsNetTarget by @nohwnd in microsoft/vstest#15523 * Add unattended work instructions to copilot-instructions.md by @nohwnd in microsoft/vstest#15531 * Reduce code style rule severity from warning to suggestion by @nohwnd in microsoft/vstest#15522 * Remove Debug/Release line number branching from tests by @nohwnd in microsoft/vstest#15519 * Revise unattended work instructions in copilot-instructions.md by @nohwnd in microsoft/vstest#15532 * Improve CompatibilityRowsBuilder error message with diagnostic details by @nohwnd in microsoft/vstest#15529 * docs: add git worktree and upstream sync workflow to copilot-instructions.md by @nohwnd in microsoft/vstest#15538 * Add VSIX runner to smoke tests by @nohwnd in microsoft/vstest#15541 * Remove deprecated WebTest and TMI test methods by @nohwnd in microsoft/vstest#15525 * Fix compatibility test failures for legacy vstest.console and MSTest adapter by @nohwnd in microsoft/vstest#15534 * Convert TestPlatform.sln to slnx format by @nohwnd in microsoft/vstest#15551 * Convert test/TestAssets .sln files to .slnx format by @nohwnd in microsoft/vstest#15557 ... (truncated) Commits viewable in [compare view](microsoft/vstest@v18.5.1...v18.8.1). </details> Updated [Microsoft.VisualStudio.Threading](https://github.com/microsoft/vs-threading) from 17.14.15 to 18.7.23. <details> <summary>Release notes</summary> _Sourced from [Microsoft.VisualStudio.Threading's releases](https://github.com/microsoft/vs-threading/releases)._ ## 18.7.23 ## What's Changed ### Fixes * Fix `CancellationToken.Combine` with 3+ cancelable tokens by @AArnott in microsoft/vs-threading#1443 * Fix VSTHRD110 firing in Expression-valued scenarios by @AArnott with @Copilot in microsoft/vs-threading#1467 * Fix super set for VSTHRD103 by @AArnott in microsoft/vs-threading#1545 * Fix VSTHRD114 not firing for null in ternary conditional expressions by @AArnott with @Copilot in microsoft/vs-threading#1548 * Disable VSTHRD010 in AppWithoutMainThread.editorconfig by @ArturDorochowicz in microsoft/vs-threading#1562 * Fix VSTHRD103 missing diagnostic for sync extension methods with async alternatives in the same static class by @drewnoakes with @Copilot in microsoft/vs-threading#1569 ### Enhancements * Add `JoinableTaskFactory.DisableProcessing()` by @AArnott in microsoft/vs-threading#1576 * Add `NoMessagePumpSyncContext..ctor(SynchronizationContext)` for Post/Send behaviors by @AArnott in microsoft/vs-threading#1578 * Add trim and NativeAOT safety attributes by @AArnott in microsoft/vs-threading#1471 * Add `IPendingExecutionRequestState` interface to expose completion state of `SingleExecuteProtector` by @lifengl in microsoft/vs-threading#1447 * Add AdditionalFiles support to VSTHRD103 analyzer for excluding specific APIs by @AArnott with @Copilot in microsoft/vs-threading#1465 * Document InvalidOperationException for AsyncReaderWriterLock acquisition methods by @AArnott with @Copilot in microsoft/vs-threading#1466 * Allow library code to detect the JoinableTaskContext is not associated with Main thread by @lifengl in microsoft/vs-threading#1477 * remove NotifyOfCrossThreadDependency call inside get_NoMessagePumpSynchronizationContext by @lifengl in microsoft/vs-threading#1486 * reduce overhead when running in no main thread mode by @lifengl in microsoft/vs-threading#1502 * Join tasks waited by JoinableTaskCollection.JoinUntilEmpty in dumpasync result by @lifengl in microsoft/vs-threading#1538 * Remove slow regex from threading analyzers by @AArnott in microsoft/vs-threading#1547 ## New Contributors * @jgrosic made their first contribution in microsoft/vs-threading#1485 * @AbhitejJohn made their first contribution in microsoft/vs-threading#1533 * @ArturDorochowicz made their first contribution in microsoft/vs-threading#1562 * @microsoft-github-policy-service[bot] made their first contribution in microsoft/vs-threading#1587 **Full Changelog**: microsoft/vs-threading@v17.14.15...v18.7.23 Commits viewable in [compare view](microsoft/vs-threading@v17.14.15...v18.7.23). </details> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore <dependency name> major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore <dependency name> minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore <dependency name>` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore <dependency name>` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore <dependency name> <ignore condition>` will remove the ignore condition of the specified dependency and ignore conditions </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: The Keeper of the Crypto Hives <235137155+cryptohivekeeper@users.noreply.github.com>
Root cause: Both
NegatableTypeOrMemberReferenceRegexandMemberReferenceRegexcontained(?<typeName>[^\[\]\:]+)+— a character-class group with an unnecessary outer + quantifier. When the overall match failed on a long type name, the regex engine tried exponentially many ways to split the type-name substring across the group, causing catastrophic backtracking.Fix: Removed both Regex fields and RegexMatchTimeout entirely from CommonInterest.cs, replacing each call site with custom string parsers.
Also add an mcp.json file.