AMD Ryzen™ AI software includes the tools and runtime libraries for optimizing and deploying AI inference on AMD Ryzen AI powered PCs1. Ryzen AI software enables applications to run on the neural processing unit (NPU) built in the AMD XDNA™ architecture, the first dedicated AI processing silicon on a Windows x86 processor2, and supports an integrated GPU (iGPU).
Developing AI applications for Ryzen AI can be summarized in 3 easy steps:
Start with a Pre-trained Model Use a pre-trained model in PyTorch or TensorFlow as your starting point. Then convert your model to the ONNX format, which is compatible with the Ryzen AI workflow.
Quantization Quantize your model by converting its parameters from floating-point to lower precision representations, like 16-bit or 8-bit integers. The Vitis™ AI Quantizer for ONNX provides an easy-to-use Post Training Quantization (PTQ) flow for this purpose.
Deploy the Model After quantization, your model is ready to be deployed on the hardware. Use ONNX Runtime with C++ or Python APIs to deploy the AI model. The Vitis AI Execution Provider included in ONNX Runtime optimizes workloads, ensuring optimal performance and lower power consumption.
Improved performance and reduced compile time for BF16 compiler
New Integer compiler with support for General Asymmetric quantization (A8W8, A16W8)
Expanded set of NPU-only models with optimized performance (higher tokens/sec on KRK and GPT)
Context length increased up to 4K for all LLMs
Added support for new LLM architectures: Phi-4 and Qwen-3
Dynamic resolution support for SD 3.0/3.5, with performance gains across other SD models including new additions (SD2.1-v 768×768 text2image, SDXL-base 1024×1024 text2image)
Unified Installer with both LLM and General Model Flow (INT8, BF16) -- a seamless experience with support for all model types in a single release package.
First release of Window support for BF16 model compilation and quantization for CNN and NLP workloads.
Support for LLM OGA Flow, making it easier to deploy LLMs efficiently.
Explore open-source tools from AMD that empower developers to analyze, optimize, and deploy AI models efficiently across diverse hardware.
Digest AI
Digest is a powerful model analysis tool designed to help you extract valuable insights from your machine learning models, enabling optimization and direct modification.
TurnkeyML simplifies the use of tools within the ONNX ecosystem by offering no-code CLIs and low-code APIs. With Turnkey, you can export and optimize ONNX models for CNNs and Transformers. With Lemonade, you can serve and benchmark LLMs on CPU, GPU, and NPU.
Ryzen™ AI technology is compatible with all AMD Ryzen™ 7040 series processors except the Ryzen™ 5 7540U and Ryzen™ 3 7440U. OEM enablement is required. Please check with your system manufacturer for feature availability prior to purchase. GD-220.
As of May 2023, AMD has the first available dedicated AI engine on an x86 Windows processor, where 'dedicated AI engine' is defined as an AI engine that has no function other than to process AI inference models and is part of the x86 processor die. For detailed information, please check: https://www.amd.com/en/technologies/xdna.html. PHX-3a
Based on testing by AMD as of 6/5/2023. Battery life results evaluated by operation of a simulated nine-participant Microsoft Teams video conference using a Ryzen™ 7940HS processor with Ryzen™ AI and integrated Radeon graphics with Windows Studio Effects vs. NVIDIA Broadcast for AI-enhanced background blur and eye gaze correction features with NVIDIA GeForce RTX 4070 discrete graphics. AMD/NVIDIA systems run from power level 100% to > 5% @150nits brightness and power mode set to "power efficiency." System configurations: Razer Blade 14” laptop, AMD Ryzen™ 9 7940HS processor with Ryzen™ AI, Integrated AMD Radeon Graphics (22.40.03.24 driver), 16GB (8GBx2) LPDDR5, NVME SSD storage, Windows 11 Home 22H , NVIDIA GeForce RTX 4070 graphics (528.92 driver) with NVIDIA Broadcast. System manufacturers may vary configurations, yielding different results. Results may vary. PHX-51
