NVIDIA Grace CPU Integrates with the Arm Software Ecosystem
The NVIDIA Grace CPU is transforming data center design by offering a new level of power-efficient performance. Built specifically for data center scale, the Grace CPU is designed to handle demanding workloads while consuming less power.
NVIDIA believes in the benefit of leveraging GPUs to accelerate every workload. However, not all workloads are accelerated. This is especially true for those workloads involving complex, branchy code such as graph analytics, commonly used in popular use cases like fraud detection, operational optimization, and social network analysis.
As data centers face increasing power constraints, it’s crucial to accelerate as many workloads as possible and run the rest on the most efficient compute possible. The Grace CPU is optimized to handle both accelerated and CPU-only tasks, delivering up to 2x the performance at the same power as conventional CPUs.
The Grace CPU features 72 high-performance and energy-efficient Arm Neoverse V2 cores, connected by the NVIDIA Scalable Coherency Fabric (SCF). This high-bandwidth fabric ensures smooth data flow between CPU cores, cache, memory, and system I/O, providing up to 3.2 TB/s of bisection bandwidth—double that of traditional CPUs.
The Grace CPU also uses high-speed LPDDR5X memory with server-class reliability, delivering up to 500 GB/s of memory bandwidth while consuming just one-fifth the energy of traditional DDR memory.
In this post, we wanted to share how the Grace CPU builds on the existing Arm ecosystem while taking advantage of the vast array of NVIDIA software and tools.
Standard software infrastructure
The Grace CPU was designed to be a balanced general-purpose CPU and to work just like any other CPU. The workflow for getting software to run on the Grace CPU is the same workflow that you’d use on any x86 CPU. Standard Linux distros (Ubuntu, RHEL, SLES, and so on) and any multi-platform, open-source compiler (GCC, LLVM, and so on) all support the Grace CPU.
The majority of open source software today already supports Arm, and thus is supported on  the Grace CPU. Similarly, any software optimizations and porting done on the Grace CPU also work on the rest of the Arm Neoverse software ecosystem.
NVIDIA continues to work with developers and partners in the Arm ecosystem and is committed to ensure that open-source compilers, libraries, frameworks, tools, and applications fully leverage Arm Neoverse-based CPUs, like the Grace CPU.
Many cloud-native and commercial ISV applications already provide optimized executables for Arm. The
Arm Developer Hub
provides a showcase of selected software packages for AI, cloud, data center, 5G, networking, and edge. This hub also provides guidance on
how to migrate
applications to Arm.
This ecosystem is enabled by Arm standards, such as the Arm Server Base System Architecture (SBSA) and the Base Boot Requirements (BBR) of the Arm SystemReady Certification Program.
NVIDIA software supports the Arm ecosystem
Arm has invested for decades in the software ecosystem. You can innovate and know that the software not only just works but is optimized for Arm. The NVIDIA software ecosystem also takes advantage of decades of work in accelerated computing and has now been optimized for Arm:
The
NVIDIA HPC SDK
and every CUDA component have Arm-native installers and
containers
.
The NVIDIA container ecosystem of NVIDIA NIM microservices and
NGC
provides deep learning, machine learning, and HPC containers optimized for Arm. NVIDIA NIM enhances inference performance, enabling high-throughput and low-latency AI at scale.
NVIDIA is also expanding its software ecosystem for Arm CPUs. NVIDIA previously launched a new suite of high performance math libraries for Arm CPUs called
NVIDIA Performance Libraries (NVPL)
. These libraries implement standard APIs, making their adoption an easy drop-in replacement from x86 at the linking stage.
Similarly, math libraries such as the
Arm’s Performance Library
(ArmPL) are also tuned to maximize the performance of the Grace CPU in addition to any other Arm CPU. For example, Arm has shared how
ArmPL Sparse can be used in a similar fashion to x86
. ArmPL has similar APIs to those of the x86 math libraries, which means that developing a wrapper may require nothing more than just a few API changes in the code.
NVIDIA is an active participant in the open-source software communities like those for GCC and LLVM compilers. If you don’t want to wait for these regular releases and want to build code that performs optimally on the Grace CPU, the latest optimizations are also made available through the
Clang distribution
.
Seamlessly moving your software to Arm
The Arm software ecosystem is large and growing, with hundreds of open source projects and commercial ISVs already supporting the Arm architecture. If your application is not yet supported, you may need to just recompile the source code. There are a variety of tools available to help you do so:
NVIDIA LaunchPad
has a module on porting to Arm that you can try for yourself.
For vector intrinsics, such as AVX, there are tools that simplify conversion.
SIMD Everywhere
and the
A Demonstration of AI and HPC Applications for NVIDIA Grace CPU
GTC session walk through several examples.
The
Arm Neoverse Migration Overview
has detailed training.
For more information about application porting and optimization, see the
NVIDIA Grace Performance Tuning Guide
. It includes instructions for setting up and optimizing performance on the Grace CPU. It also provides high-level developer guidance on Arm SIMD programming, the Arm memory model, and other details. Use this guide to help you realize the best possible performance for your particular NVIDIA Grace system.
Figure 1. Running software on the Grace CPU uses the same process to optimize as for any other CPU
Summary
The NVIDIA Grace CPU is designed for the modern data center with 72 high-performance Arm Neoverse V2 cores, an NVIDIA-designed high-bandwidth SCF to maximize performance and high-bandwidth low-power memory. It can deliver up to 2x the performance in the same power envelope as leading traditional x86 CPUs.
The NVIDIA Grace CPU is a standards-based Arm SBSA design that works just like any other CPU and is fully compatible with the broad Arm software ecosystem.
For more information about software and system setup, see
NVIDIA Grace CPU
.
NVIDIA Grace CPU 与 Arm 软件生态系统集成
NVIDIA Grace CPU 将节能高效的性能提升到了新的高度，正在改变数据中心的设计。Grace CPU 专为数据中心规模打造，旨在处理要求严苛的工作负载，同时降低功耗。
NVIDIA 相信利用 GPU 加速各种工作负载的好处。但是，并非所有工作负载都得到加速。对于涉及复杂的分支代码 (例如图形分析) 的工作负载尤其如此，这些代码通常用于欺诈检测、运营优化和社交网络分析等热门用例。
随着数据中心面临越来越大的功率限制，加速尽可能多的工作负载并在尽可能高效的计算上运行其余工作负载变得至关重要。Grace CPU 经过优化，可处理加速任务和仅使用 CPU 的任务，在功耗与传统 CPU 相同的情况下提供高达 2 倍的性能。
Grace CPU 配备 72 个高性能、高能效的 Arm Neoverse V2 核心，由 NVIDIA Scalable Coherency Fabric (SCF) 连接。这种高带宽结构可确保在 CPU 核心、缓存、内存和系统 I/O 之间实现流畅的数据流，并提供高达 3.2 TB/s 的对分带宽，是传统 CPU 带宽的两倍。
Grace CPU 还采用具有服务器级可靠性的高速 LPDDR5X 内存，提供高达 500 GB/s 的内存带宽，而能耗仅为传统 DDR 内存的五分之一。
在本文中，我们想分享 Grace CPU 如何在现有 Arm 生态系统的基础上构建，同时利用大量 NVIDIA 软件和工具。
标准软件基础架构
Grace CPU 被设计为平衡型通用 CPU，与其他 CPU 一样工作。在 Grace CPU 上运行软件的工作流与在任何 x86 CPU 上使用的工作流相同。标准 Linux 发行版 (Ubuntu、RHEL、SLES 等) 和任何多平台开源编译器 (GCC、LLVM 等) 都支持 Grace CPU。
如今，大多数开源软件已经支持 Arm，因此 Grace CPU 也支持这些软件。同样，在 Grace CPU 上完成的任何软件优化和移植也适用于 Arm Neoverse 软件生态系统的其余部分。
NVIDIA 将继续与 Arm 生态系统中的开发者和合作伙伴合作，并致力于确保开源编译器、库、框架、工具和应用充分利用基于 Arm Neoverse 的 CPU (例如 Grace CPU)。
许多云原生和商用 ISV 应用已经为 Arm 提供了优化的可执行文件。Arm 开发者中心展示了适用于 AI、云、数据中心、5G、网络和边缘的精选软件包。该中心还提供了有关
如何将应用程序迁移到 Arm 的指导
。
此生态系统由 Arm 标准提供支持，例如 Arm Server Base System Architecture（SBSA）和 Arm SystemReady 认证计划的 Base Boot Requirements（BBR）。
NVIDIA 软件支持 Arm 生态系统
数十年来，Arm 一直在软件生态系统方面进行投资。您可以不断创新，并且知道该软件不仅有效，还针对 Arm 进行了优化：NVIDIA 软件生态系统还利用数十年来在加速计算方面的工作成果，现已针对 Arm 进行了优化：
NVIDIA HPC SDK
和每个 CUDA 组件都具有 Arm 原生安装程序和
容器
。
NVIDIA 容器生态系统的 NVIDIA NIM 微服务和
NGC
提供深度学习、机器学习和 HPC 容器，优化为 Arm。NVIDIA NIM 可增强推理性能，实现高吞吐量和低延迟的大规模 AI。
NVIDIA 还在扩展面向 Arm CPU 的软件生态系统。NVIDIA 之前为 Arm CPU 推出了一套新的高性能数学库，称为
NVIDIA Performance Libraries (NVPL)
。这些库实现了标准 API，因此在链接阶段采用这些 API 可以轻松替代 x86，
同样，
Arm 的性能库（ArmPL）
等数学库也进行了调整，以更大限度地提高 Grace CPU 以及任何其他 Arm CPU 的性能。例如，Arm 分享了如何以与 x86 类似的方式使用
ArmPL Sparse。ArmPL 具有与 x86 数学库类似的 API
，这意味着开发包装器只需要更改代码中的几个 API 即可。
NVIDIA 积极参与开源软件社区，例如 GCC 和 LLVM 编译器社区。如果您不想等待这些常规版本的发布，并且想要构建在 Grace CPU 上以最佳性能执行的代码，您还可以通过 Clang 发行版获得最新的优化。
将您的软件无缝迁移到 Arm
Arm 软件生态系统规模庞大且不断发展，已有数百个开源项目和商业 ISV 支持 Arm 架构。如果应用程序尚不受支持，您可能需要重新编译源代码。有多种工具可帮助您实现这一目标：
NVIDIA LaunchPad
提供了一个移植到 Arm 的模块，您可以自行试用。
对于 AVX 等向量内部函数，有一些工具可以简化转换。
SIMD Everywhere
和
NVIDIA Grace CPU 的 AI 和 HPC 应用演示
GTC 会议将介绍几个示例。
Arm Neoverse 迁移概述提供了详细的训练。
有关应用程序移植和优化的更多信息，请参阅
NVIDIA Grace Performance Tuning Guide
。其中包含有关在 NVIDIA Grace CPU 上设置和优化性能的说明。它还为开发者提供有关 Arm SIMD 编程、Arm 内存模型和其他详细信息的高级指导。使用本指南帮助您实现特定 NVIDIA Grace 系统的最佳性能。
图 1、在 Grace CPU 上运行软件会使用与任何其他 CPU 相同的流程进行优化
总结
NVIDIA Grace CPU 专为现代数据中心而设计，拥有 72 个高性能 Arm Neoverse V2 核心，这是 NVIDIA 设计的高带宽 SCF，可更大限度地提高性能和高带宽低功耗内存。与领先的传统 x86 CPU 相比，它可以在相同的功率范围内提供高达 2 倍的性能。
NVIDIA Grace CPU 是基于标准的 Arm SBSA 设计，其工作方式与任何其他 CPU 相同，并且与广泛的 Arm 软件生态系统完全兼容。
有关软件和系统设置的更多信息，请参阅
NVIDIA Grace CPU
。