Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

[MLAS] Adding DynamicQGemm function pointers and ukernel interface - #27403

#27403
Merged
hariharans29 merged 1 commit into
microsoft:mainmicrosoft/onnxruntime:mainfrom
patryk-kaiser-ARM:qgemm_pointerspatryk-kaiser-ARM/onnxruntime:qgemm_pointersCopy head branch name to clipboard
Feb 21, 2026
Merged

[MLAS] Adding DynamicQGemm function pointers and ukernel interface#27403
hariharans29 merged 1 commit into
microsoft:mainmicrosoft/onnxruntime:mainfrom
patryk-kaiser-ARM:qgemm_pointerspatryk-kaiser-ARM/onnxruntime:qgemm_pointersCopy head branch name to clipboard

Conversation

@patryk-kaiser-ARM

Copy link
Copy Markdown
Contributor

Description

  • Adding function pointer overrides to KleidiAI DynamicQGemm
  • Making use of ukernel interface for DynamicQGemm to select between SME and SME2 variants

Motivation and Context

Fixes #26377

@patryk-kaiser-ARM

Copy link
Copy Markdown
Contributor Author

@microsoft-github-policy-service agree company="Arm"

@hariharans29

Copy link
Copy Markdown
Member

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 4 pipeline(s).

@hariharans29

Copy link
Copy Markdown
Member

Also fixes related issue #26361

@hariharans29
hariharans29 enabled auto-merge (squash) February 20, 2026 23:12
@hariharans29
hariharans29 merged commit df9146f into microsoft:main Feb 21, 2026
88 checks passed
tianleiwu pushed a commit that referenced this pull request Feb 26, 2026
…27403)

### Description
* Adding function pointer overrides to KleidiAI DynamicQGemm
* Making use of ukernel interface for DynamicQGemm to select between SME
and SME2 variants

### Motivation and Context
Fixes #26377
tianleiwu added a commit that referenced this pull request Feb 27, 2026
This cherry-picks the following commits for the release:

| Commit ID | PR Number | Commit Title |
|-----------|-----------|-------------|
| decd177 | #27090 | Fix GatherND division by zero when batch
dimensions mismatch |
| 55f8234 | #27360 | Fix QMoE CPU Operator |
| df9146f | #27403 | [MLAS] Adding DynamicQGemm function pointers and
ukernel interface |
| 0f93853 | #27318 | [js/web] Use embedded WASM module in Blob URL
workers when wasmBinary is provided |
| b2a6e69 | #27364 | QMoE CPU Performance Update (Up to 4x on 4-bit)
|
| f501e1d | #27413 | Fix refcount bug in map input conversion that
caused shutdown segfault |
| b32b205 | #27421 | Fix error where bytes is not assigned for
dynamic qgemm pack b size |
| 426b006 | #27397 | Fix DllImportResolver |
| 0982844 | #27412 | MatmulNBits prepacking scales fix |
| 9afb0d2 | #27430 | Fix validation for external data paths for
models loaded from bytes |
| 71d2cd0 | #27401 | Enable Python 3.14 CI and Upgrade Dependencies |
| 79e0676 | #27419 | fix: out of bounds access for resize operation |
| 82eb99c | #27459 | Fix SkipLayerNorm fusion incorrectly applied
when gamma/beta are not 1D |
| 355278a | #27444 | Fix GatherCopyData Integer Truncation Leading to
Heap Out-of-Bounds Read/Write |
| cf96123 | #27411 | [web] fix usage of wasmBinary together with a
blob URL for .mjs |
| 1131a86 | #27399 | [web] remove the unhelpful "Unknown CPU vendor"
warning. |
| ffbbc4f | #27316 | Build Windows ARM64X binaries as part of
packaging pipeline |

---------

Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com>
Co-authored-by: patryk-kaiser-ARM <patryk.kaiser@arm.com>
Co-authored-by: don <70039285+0-don@users.noreply.github.com>
Co-authored-by: Jonathan Clohessy <jonathan.clohessy@arm.com>
Co-authored-by: Hariharan Seshadri <shariharan91@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Adrian Lizarraga <adlizarraga@microsoft.com>
Co-authored-by: Lukas Folle <126877803+lukas-folle-snkeos@users.noreply.github.com>
Co-authored-by: Chi Lo <54722500+chilo-ms@users.noreply.github.com>
Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com>
Co-authored-by: Chaya <cha182350@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Erik <erscor@microsoft.com>
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KleidiAI library usage needs to check for SME2, not SME

3 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.