Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

arm64: Improve predicate instruction usage - #129506

#129506
Open
jonathandavies-arm wants to merge 5 commits into
dotnet:maindotnet/runtime:mainfrom
jonathandavies-arm:upstream/sve/predicate-instruction-usagejonathandavies-arm/runtime:upstream/sve/predicate-instruction-usageCopy head branch name to clipboard
Open

arm64: Improve predicate instruction usage#129506
jonathandavies-arm wants to merge 5 commits into
dotnet:maindotnet/runtime:mainfrom
jonathandavies-arm:upstream/sve/predicate-instruction-usagejonathandavies-arm/runtime:upstream/sve/predicate-instruction-usageCopy head branch name to clipboard

Conversation

@jonathandavies-arm

Copy link
Copy Markdown
Contributor

SVE Predicate Instructions asm diffs

Branch: upstream/sve/predicate-instruction-usage

Collection / JIT / output paths

Collection

  • MCH: artifacts/asmdiff/custom_mch/sve_predicate_instructions.mch
  • Source test DLL: artifacts/tests/coreclr/linux.arm64.Checked/JIT/opt/SVE/PredicateInstructions/PredicateInstructions.dll

JITs

  • Base: artifacts/asmdiff/builds/5656cc9c/core_root/libclrjit.so
  • Diff: artifacts/bin/coreclr/linux.arm64.Checked/libclrjit.so

Output

  • Dasm base: artifacts/spmi/asm.sve_predicate_instructions/base/
  • Dasm diff: artifacts/spmi/asm.sve_predicate_instructions/diff/
  • Log: artifacts/spmi/superpmi.2.log

Summary

3 asm diffs were found, all in PredicateInstructions.

Context Method Code bytes PerfScore Bytes allocated
10 AndMask(Vector<short>, Vector<short>) 52 -> 48 (-4) 23.50 -> 21.50 (-2.00) 60992 -> 60232 (-760)
11 BitwiseClearMask(Vector<short>, Vector<short>) 44 -> 40 (-4) 18.00 -> 17.50 (-0.50) 59760 -> 59544 (-216)
16 TransposeEvenAndMask(Vector<short>, Vector<short>, Vector<short>) 52 -> 52 (+0) 24.50 -> 24.50 (+0.00) 62264 -> 61360 (-904)

Notable diffs

Context 10: AndMask

Removed a redundant ptrue and now uses the first predicate as the governing predicate for and:

- ptrue   p2.h
- and     p0.b, p2/z, p0.b, p1.b
+ and     p0.b, p0/z, p0.b, p1.b
Context 11: BitwiseClearMask

Uses predicate bic directly instead of materializing predicate results into vectors and doing vector bic:

- mov     z16.h, p0/z, #1
- ptrue   p0.h
- cmpeq   p0.h, p0/z, z0.h, z1.h
+ ptrue   p1.h
+ cmpeq   p1.h, p1/z, z0.h, z1.h
+ bic     p0.b, p0/z, p0.b, p1.b
  mov     z0.h, p0/z, #1
- bic     v0.8h, v16.8h, v0.8h
Context 16: TransposeEvenAndMask

Removes one ptrue p2.h; total code size and PerfScore are unchanged:

- ptrue   p2.h
- and     p0.b, p2/z, p0.b, p1.b
- cmpgt   p1.h, p2/z, z1.h, z2.h
+ and     p0.b, p0/z, p0.b, p1.b
+ ptrue   p1.h
+ cmpgt   p1.h, p1/z, z1.h, z2.h

@github-actions github-actions Bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jun 17, 2026
@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Jun 17, 2026
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

@dhartglassMSFT

Copy link
Copy Markdown
Contributor

@a74nh does this change depend on the predicate-discussion in #128326 , or am I misremembering our conversation?

@a74nh

a74nh commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

@a74nh does this change depend on the predicate-discussion in #128326 , or am I misremembering our conversation?

We didn't mention this in our discussion, but, yes I think we should hold of from merging this until #128326 is complete.

@github-actions

github-actions Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Workflow state for the Holistic Review Orchestrator.

{
  "version": 5,
  "last_dispatched_commit": "35eb3b6ab6b6b2f3a184e525eaf24e704c2c926f",
  "last_dispatched_base_ref": "main",
  "last_dispatched_base_sha": "1af2b9fda524303a30c5d14ac47bbf5510e06102",
  "last_reviewed_commit": "35eb3b6ab6b6b2f3a184e525eaf24e704c2c926f",
  "last_reviewed_base_ref": "main",
  "last_reviewed_base_sha": "1af2b9fda524303a30c5d14ac47bbf5510e06102",
  "last_recorded_worker_run_id": "29677704668",
  "review_attempt_commit": "",
  "review_attempt_base_ref": "",
  "review_attempt_count": 0,
  "max_review_attempts": 5,
  "review_history_format": "holistic-review-disclosure-v1",
  "review_history": [
    {
      "commit": "35eb3b6ab6b6b2f3a184e525eaf24e704c2c926f",
      "review_id": 4730527203
    }
  ]
}

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Holistic Review

Motivation: The motivation is real and well-evidenced. On SVE, boolean masks are naturally predicates, but the JIT currently materializes predicate results into vectors (mov z, p/z, #1) and re-compares them to zero before doing logical ops, then compares back. Recovering the predicate form lets the JIT emit and/bic/orr/eor/sel/zip1 directly on predicate registers and reuse the first predicate as the governing predicate, removing redundant ptrue/mov sequences. The PR's own SPMI diffs show real code-size and allocation wins.

Approach: The approach is reasonable and consistent with existing patterns: recognize mask-shaped operands (ConvertMaskToVector, real masks, vector zero, and constant 0/1 vectors) during gtFoldExprHWIntrinsic and in arm64 lowering, and fold to the _Predicates intrinsic variants. Importing Sve.BitwiseClear as AND(NOT(...)) and recovering bic in lowering mirrors the established AdvSimd handling. The codegen using op1Reg as the governing predicate matches the documented and Pd, Pn/z, Pn, Pm semantics.

Summary: ⚠️ Needs Changes. The direction and most of the implementation look sound, but TryLowerSvePredicateBitwiseClear ends with an unconditional return true; in its fall-through path. Since the helper now runs for every arm64 vector GT_AND, a true result when no predicate pattern matched causes the caller to break and skip the pre-existing AdvSimd ~op1 & op2 → BitwiseClear (AndNot) folding for ordinary vector ANDs. This is a functional regression outside the intended SVE-predicate scope and is not covered by the added SVE-only tests. See the inline comment for details.


Detailed Findings

❌ Correctness — Unconditional return true; disables AdvSimd AndNot folding

Flagged inline at src/coreclr/jit/lowerarmarch.cpp fall-through return. The final return true; should be return false; so that non-predicate vector ANDs fall through to the existing NI_AdvSimd_BitwiseClear/AndNot recognition in LowerHWIntrinsic. As written, that established optimization is bypassed for all arm64 vector GT_AND whose operands are not mask-shaped.

⚠️ Test coverage — No coverage for the non-predicate AndNot path

The added tests in PredicateInstructions.cs exercise only the new SVE-predicate folds (all inputs mask-shaped). None exercise an ordinary vector ~a & b / Vector*.AndNot on arm64, which is exactly the path broken by the finding above. A regression test covering that AdvSimd AndNot lowering would prevent this class of regression.

✅ Codegen and fold logic — Predicate variants look correct

The NI_Sve_And_Predicates/NI_Sve_BitwiseClear_Predicates codegen reusing op1Reg as the governing predicate matches the SVE semantics in the PR description, and the 0/1-vector-to-mask recognition in gtFoldExprHWIntrinsic is guarded to arm64 + masked HW intrinsics and rejects non-0/1 constants. TryLowerSveConvertVectorToMask correctly only elides the conversion when the source is already a mask.

Note

This review was generated by this repository's Holistic Review agentic workflow to complement the built-in Copilot review.

Generated by Holistic Review · 141.4 AIC · ⌖ 10.7 AIC · ⊞ 10K

Comment thread src/coreclr/jit/lowerarmarch.cpp Outdated
return true;
}

return true;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fall-through return true; looks like a bug. When none of the three predicate patterns match (the common case for ordinary vector GT_AND), the function still returns true. The caller does:

if ((oper == GT_AND) && TryLowerSvePredicateBitwiseClear(node, &intrinsicId, &oper))
{
    break;
}

so a true result breaks out of the GT_AND/GT_OR case and skips the pre-existing AdvSimd folding that turns ~op1 & op2 into NI_AdvSimd_BitwiseClear (AndNot). Because this helper is now invoked for every arm64 vector GT_AND, that established AndNot recognition is effectively disabled for all non-predicate vector ANDs whenever the operands are not mask-shaped — a functional regression well beyond the intended SVE-predicate scope.

This should almost certainly return false; here so the caller falls through to the existing AdvSimd transform. (A regression test exercising e.g. Vector128.AndNot / ~a & b on arm64 would catch this; the current SVE-only tests do not.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've fixed the return true issue.

@dhartglassMSFT

Copy link
Copy Markdown
Contributor

@jonathandavies-arm I'll take a look here now that #128326 has gone in, though it looks like there's merge conflicts

…e-instruction-usage

# Conflicts:
#	src/coreclr/jit/hwintrinsicarm64.cpp
#	src/coreclr/jit/lowerarmarch.cpp
#	src/tests/JIT/opt/SVE/PredicateInstructions.cs
@dhartglassMSFT

Copy link
Copy Markdown
Contributor

Failures in conditionalselect tests

error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_byte() has failed.
error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_ushort() has failed.
error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_uint() has failed.
error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_ulong() has failed.
error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_sbyte() has failed.
error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_short() has failed.
error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_long() has failed.
Sve.CreateBreakPropagateMask<Byte>(Vector<Byte>, Vector<Byte>): ConditionalSelectScenario_TrueValue failed:
    mask: (0, 0, 0, 0, 0, 104, 70, 174, 0, 67, 0, 134, 240, 53, 189, 13)
    left: (238, 200, 173, 46, 225, 123, 187, 87, 221, 76, 202, 176, 135, 211, 49, 163)
   right: (164, 121, 67, 158, 115, 24, 36, 113, 253, 7, 51, 237, 3, 122, 156, 158)
 falseOp: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
  result: (255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255)

@jonathandavies-arm

Copy link
Copy Markdown
Contributor Author

Failures in conditionalselect tests

error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_byte() has failed.
error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_ushort() has failed.
error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_uint() has failed.
error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_ulong() has failed.
error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_sbyte() has failed.
error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_short() has failed.
error : Test ____w_1_s_src_tests_JIT_HardwareIntrinsics_Arm_Sve_Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_CreateBreakPropagateMask_long() has failed.
Sve.CreateBreakPropagateMask<Byte>(Vector<Byte>, Vector<Byte>): ConditionalSelectScenario_TrueValue failed:
    mask: (0, 0, 0, 0, 0, 104, 70, 174, 0, 67, 0, 134, 240, 53, 189, 13)
    left: (238, 200, 173, 46, 225, 123, 187, 87, 221, 76, 202, 176, 135, 211, 49, 163)
   right: (164, 121, 67, 158, 115, 24, 36, 113, 253, 7, 51, 237, 3, 122, 156, 158)
 falseOp: (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
  result: (255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255)

This should be fixed now. I'll keep an eye on the tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arch-arm64 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.