[Perf] Windows/x64: 1 Regression on 5/27/2026 2:58:50 PM +00:00

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.22631
Queue	ViperWindows
Baseline	ac7b8a0391e871a3b07bec217d87cf275423016f
Compare	2a546738e968e3f75092657d4bee6bd21821e069
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Threading.Tests.Perf_Timer

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
AsynchronousContention - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	16.80 secs	21.45 secs	1.28	0.76	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Timer*'

Details

System.Threading.Tests.Perf_Timer.AsynchronousContention

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[LoopedBard3]

🔍 Automated Triage Analysis

Summary: Verified regression in AsynchronousContention — caused by commit 0ffb238 ("Rework AsyncHelpers await helpers for correctness and efficiency (#128528)").

Finding Confidence: 5/5 (analysis accuracy)
Regression Confidence: 5/5 (likelihood of true regression)

Likely Cause: 0ffb2389feb41141fdfc957ef1be4bca7e92ccb9

🤖 Proposed Next Actions: Known-cause regression — transfer & notify — 4 proposed (rule R4_known_regression_new — see Full Analysis for the table).

📊 Full Analysis (click to expand)

Summary: Clear regression in System.Threading.Tests.Perf_Timer.AsynchronousContention (+27.6%) attributed to commit 0ffb238 ("Rework AsyncHelpers await helpers for correctness and efficiency"). Finding confidence 4/5 (strong historical data, 300 runs, consistent step change across 23 post-regression runs). Regression confidence 4/5 (direct code path hit via Task.Yield/async continuations, step change confirmed, small commit range of 11 commits).

Issue Overview

• Issue: [Perf] Windows/x64: 1 Regression on 5/27/2026 2:58:50 PM +00:00 #128908 - [Perf] Windows/x64: 1 Regression on 5/27/2026 2:58:50 PM +00:00
• Date: 2026-05-27 | Queue: Windows.11.Amd64.Viper.Perf | OS/Arch: Windows/x64
• Commits: ac7b8a0391e8 → 2a546738e968 (11 commits)
• Labels: untriaged, perf-regression, os-windows, arch-x64, branch-refs/heads/main, runtime-coreclr, kind-micro, compilationmode-tiered, runkind-micro
• Historical Data: Available (300 runs before, 23 runs after regression)
• Runtime Stack: Desktop + CoreCLR (implicit)

Test Analysis

Test Name	Baseline	Compare	Δ%	μ	σ	n	z	Thresh%	Noise?	Assessment
System.Threading.Tests.Perf_Timer.AsynchronousContention	16.804	21.449	+27.6%	~17.36B*	~2.17B*	300	N/A*	24.95%	N	Step change, confirmed

Legend: μ=mean, σ=stddev, n=runs, z=z-score, Thresh%=adaptive threshold

*Note: Historical series values are in nanoseconds (billions range) while baseline/compare are reported in seconds. z-score is invalid due to unit mismatch but classification is confirmed by percentDelta exceeding adaptive threshold and consistent step change in post-regression cluster (23 runs).

Finding Confidence: 4/5 - Strong historical data (300 prior runs), consistent step change visible across 23 post-regression runs. High CV (12.5%) introduces some noise but the 27.6% delta clearly exceeds the adaptive threshold (24.95%). Unit mismatch in historical data prevents z-score validation but does not affect classification.

Candidate Stack Relevance

Commit	Touched Paths (summary)	Relevance	Notes
0ffb2389feb4	src/coreclr/System.Private.CoreLib/.../AsyncHelpers*.cs, src/coreclr/jit/, src/coreclr/interpreter/	Relevant	Reworks async continuation mechanics in CoreCLR
25c4ee746e35	src/coreclr/jit/objectalloc.cpp	Relevant	JIT escape analysis cleanup - unlikely to affect timer contention
38d408d22a64	src/coreclr/jit/optimizer.cpp	Relevant	JIT loop inversion optimization - unlikely for async patterns
5374890bb36a	src/coreclr/debug/crashreport/	Relevant	Diagnostics crash report - no runtime perf impact expected
d78013e4cb65	src/libraries/System.Net.Quic/	Shared	QUIC stream changes - unrelated to timer/async paths

Confirmed Regressions

System.Threading.Tests.Perf_Timer.AsynchronousContention (+27.6%)

• Delta: 16.804s → 21.449s (+27.6%)
• Historical Context: 300 prior runs; adaptive threshold=24.95%; CV=12.5%; step change confirmed with 23 post-regression runs averaging ~21.7B ns vs ~17.5B ns pre-regression (+24%)
• Assessment: Step change — sudden jump coinciding with commit range, consistent across all post-regression runs
• Finding Confidence: 4/5 - Strong historical data with 300 prior runs. High CV (12.5%) lowers precision but the consistent post-regression cluster provides clear confirmation. Unit mismatch prevents z-score usage.
• Regression Confidence: 4/5 - Direct code path hit (async continuations via Task.Yield), small commit range (11 commits), clear step change pattern. Only one dominant candidate with directly relevant file changes.
• Likely Cause: 0ffb2389feb4 - "Rework AsyncHelpers await helpers for correctness and efficiency (Rework AsyncHelpers await helpers for correctness and efficiency #128528)"
  • Affected Areas: Async infrastructure / Task continuations / CoreCLR runtime
  • Stack Relevance: Relevant
  • Rationale: This commit extensively reworks AsyncHelpers.CoreCLR.cs (+224/-111 lines), ValueTaskSourceContinuation (+16/-31), and adds a new RuntimeAsyncTaskContinuation.cs (+176 lines). The benchmark creates multiple concurrent tasks that each call await Task.Yield() inside a timer allocation loop — directly exercising the async continuation path that was reworked. The PR title mentions "correctness and efficiency" suggesting behavioral changes that may introduce additional synchronization overhead in contention scenarios. The commit also touches the JIT compiler and interpreter async support, indicating a broad async infrastructure change.
  • Candidates Considered:
    • 0ffb2389feb4 - Directly reworks async continuation machinery exercised by test
    • 38d408d22a64 - JIT loop optimization, less likely for async contention pattern
    • 25c4ee746e35 - JIT escape analysis cleanup, minimal code removal
• Verification Status: Not verified — bisection or RunWithBuildAtHash recommended to confirm

Noise Tests

None — the single reported test is classified as a true regression.

Open Questions

• Historical Data: Unit mismatch between raw baseline/compare values (seconds) and historical series (nanoseconds in billions range) prevents direct z-score calculation. Classification was based on percentDelta vs adaptive threshold and visual step change in post-regression cluster.
• Confidence Limitations: High CV (12.5%) means individual runs vary significantly, but the 23-run post-regression cluster is internally consistent enough to confirm the step change.
• Verification Needed: Verification via RunWithBuildAtHash comparing baseline commit vs 0ffb238 would elevate confidence to 5/5. Given the direct code path alignment, this is likely to confirm.

Recommended Actions

1. Verification: Verify regression with RunWithBuildAtHash on commit 0ffb2389feb4 vs baseline ac7b8a0391e8 using test System.Threading.Tests.Perf_Timer.AsynchronousContention
2. Source review: Review PR Rework AsyncHelpers await helpers for correctness and efficiency #128528 for async continuation contention changes — specifically examine whether new locking or synchronization was introduced in hot paths exercised by Task.Yield() under concurrent timer allocation
3. Contact author: Engage with the PR author regarding the performance trade-off between correctness fixes and contention overhead in AsyncHelpers

Bisection Results

Test	Status	Culprit Commit	Relevance	Reported Diff (issue)	Bisected Diff (at culprit)	Duration
AsynchronousContention	✅ Success	0ffb2389feb4	✅ RELATED	16.8→21.4 s (+28%)	15.3→20.2 s (+32%)	3654s

Total bisection time: 3654s
Success rate: 1/1 tests

System.Threading.Tests.Perf_Timer.AsynchronousContention

• Culprit: 0ffb2389feb4
• Commit Message: Rework AsyncHelpers await helpers for correctness and efficiency (Rework AsyncHelpers await helpers for correctness and efficiency #128528)
• Relevance: RELATED
• Explanation: This commit fundamentally reworks async task continuations in System.Threading.Tasks.Task.cs and AsyncHelpers, changing how continuations are registered and invoked. The Perf_Timer.AsynchronousContention test directly measures timer behavior under async contention, exercising the exact continuation scheduling paths modified by this commit.

Helix jobs

Job ID	Submitted by bisection	Last status
c8546e18-e979-4c14-bfbd-03513f56e664	✅	submitted
7bd28bd0-ec4d-487c-97a5-c7c09e283213	✅	submitted
cea11f89-c0a6-4e68-8393-cc26f7133ee3	✅	completed

Per-commit measurements

Commit	Position	Measurements
ac7b8a0391e8	baseline	15.257 s
d78013e4cb65	index 1	20.693 s
843084e7aaa9	index 2	20.004 s
7d2cb0624185	index 5	21.076 s
50b8d1a7b7bb	index 8	21.421 s
0ffb2389feb4	culprit	20.182 s

Bisection details

…[earlier output omitted]

Excellent! The baseline commit shows 15.26 s, confirming a clear step change to 20.18 s at the first commit 0ffb2389. The culprit is 0ffb2389feb41141fdfc957ef1be4bca7e92ccb9. Let me examine it.

● get_single_commit_summary (MCP: PerfTriageMcp) · owner: "dotnet", repo: "runtime", sha: "0ffb2389…
└ {"index":null,"sha":"0ffb2389feb41141fdfc957ef1be4bca7e92ccb9","abbrevSh...

I have all the data needed. The culprit is clearly 0ffb2389feb41141fdfc957ef1be4bca7e92ccb9 — it reworks async task continuations and directly modifies System.Threading.Tasks.Task.cs, which is the core infrastructure exercised by the AsynchronousContention timer test.

Here are the complete results:

BISECTION RESULT:
- Culprit Commit: 0ffb2389feb41141fdfc957ef1be4bca7e92ccb9
- Commit Message: Rework `AsyncHelpers` await helpers for correctness and efficiency (#128528)
- Commits Tested: 6
- Confidence: 5/5
- Commit Relevance: RELATED
- Relevance Explanation: This commit fundamentally reworks async task continuations in System.Threading.Tasks.Task.cs and AsyncHelpers, changing how continuations are registered and invoked. The Perf_Timer.AsynchronousContention test directly measures timer behavior under async contention, exercising the exact continuation scheduling paths modified by this commit.
- Baseline Value: 15.257 s
- Culprit Value: 20.182 s
- Summary: Bisection identified the first commit after baseline as the culprit. The baseline (ac7b8a03) measured 15.26s while the immediate next commit (0ffb2389) jumped to 20.18s (+32%), confirming the regression was introduced by the async continuations rework.

HELIX JOBS:
- c8546e18-e979-4c14-bfbd-03513f56e664
- 7bd28bd0-ec4d-487c-97a5-c7c09e283213
- cea11f89-c0a6-4e68-8393-cc26f7133ee3

MEASUREMENTS:
- ac7b8a0391e8 (baseline): 15.257 s
- 0ffb2389feb4 (culprit): 20.182 s
- d78013e4cb65 (index 1): 20.693 s
- 843084e7aaa9 (index 2): 20.004 s
- 7d2cb06241855 (index 5): 21.076 s
- 50b8d1a7b7bb (index 8): 21.421 s

🤖 Proposed Next Actions

Rule: R4_known_regression_new — Known-cause regression — transfer & notify.

Verified regression attributed to commit 0ffb238 (finding 5/5, regression 5/5, bisection RELATED). No prior linked autofile issue found.

#	Action	Status	Details
1	Back-reference cause PR	🕒 Planned	on culprit PR for 0ffb2389feb4
2	Transfer issue	🕒 Planned	→ dotnet/runtime
3	Ping owners	🕒 Planned	no owners resolved
4	Remove label	🕒 Planned	untriaged

---

Generated by PerfTriageAgent on 2026-06-02 10:30 UTC

[DrewScoggins]

Likely cause from bisection results. #128528

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @VSadov
See info in area-owners.md if you want to be subscribed.