CUDA Toolkit Now Available for NVIDIA Blackwell
The latest release of the
CUDA Toolkit
, version 12.8, continues to push accelerated computing performance in data sciences, AI, scientific computing, and computer graphics and simulation, using the latest NVIDIA CPUs and GPUs. This post highlights some of the new features and enhancements included with this release:
NVIDIA Blackwell architecture support
CUDA Graphs conditional nodes enhancements
Blackwell CUTLASS kernels for large language models (LLMs)
NVIDIA Nsight Developer Tools updates
Math libraries updates
cudaStreamGetDevice
Compiler updates
Accelerated Python updates
Feature-complete architectures
NVIDIA Blackwell architecture support
CUDA Toolkit 12.8 is the first version of the Toolkit to support the
NVIDIA Blackwell
architecture across the entire suite of Developer Tools including performance tools and profilers, libraries, and compilers. Built with 208 billion transistors—more than 2.5x the number of transistors in
NVIDIA Hopper
GPUs—Blackwell is the largest GPU ever built.
Key Blackwell capabilities supported include:
Second-generation Transformer Engine through custom Tensor Core technology:
Accelerates inference and training for LLMs and mixture-of-experts (MoE) models.
Decompression:
Accelerates performance on data analytics and data science pipelines using the latest compression formats such as LZ4, Snappy, and Deflate.
Network interconnect:
NVLink and NVLink Switches accelerate inter-GPU communications performance for trillion-parameter and multitrillion-parameter AI models.
To learn more about the leading innovations in Blackwell, see the
NVIDIA Blackwell Architecture Technical Brief
.
2x faster CUDA Graphs with runtime kernel selection for lower latency inference
With Blackwell, CUDA Graphs APIs continue to be the most efficient way to launch repeated invocations of sequences of GPU operations. CUDA Toolkit 12.8 introduces more enhancements to CUDA Graphs, including additional
conditional node
types.
In many applications, having dynamic control over the execution of work in CUDA Graphs can increase performance and flexibility of graph launches. For example, an algorithm that involves iterating over a series of operations many times until the result converges below a certain threshold can now run wholly on the GPU without needing CPU control management, reducing overhead by as much as 2x. CUDA Toolkit 12.8 improves APIs for runtime control of conditional graph nodes.
Conditional nodes contain segments of a graph that can execute, or be skipped, based on a condition to evaluate as the graph is running. Such segments can be evaluated once (an IF node), or repeatedly in a loop (a WHILE node). CUDA 12.8 adds support for two new types of conditional graph nodes: IF/ELSE combined nodes and SWITCH nodes.
With the Blackwell architecture, we’ve improved LLM performance to benefit all reasoning models, including DeepSeek-R1. CUDA Graphs enhanced SWITCH and IF/ELSE support delivers 2x more performance for runtime kernel selection versus going back to the CPU for launch decision-making.
Training
: By reducing CPU dependency for kernel selection, training workloads sustain even more GPU Tensor Core throughput, resulting in higher Model FLOPs Utilization (MFU). This improves performance using the same GPU infrastructure, reducing time and cost to train.
Inference
: For next-generation reasoning models that make use of test-time compute, a high token generation rate is critical as each inference request can generate a vast number of tokens per query. CUDA 12.8 new stream API enables fewer calls back to the host CPU, reducing the time between one kernel finishing and the next one starting, increasing token generation rate. This results in more tokens generated in fixed time budget, helping models reason more and increasing intelligence.
To learn more, see
Dynamic Control Flow in CUDA Graphs with Conditional Nodes
.
Blackwell CUTLASS kernels for LLMs
CUTLASS
, since its 2017 debut, has been instrumental for researchers and developers implementing high-performance CUDA kernels on NVIDIA GPUs. By providing developers with comprehensive tools to design custom operations, such as GEMMs and Convolutions, CUTLASS has been critical for the development of hardware-aware algorithms, powering breakthroughs like FlashAttention that helped spark modern AI.
With the release of CUTLASS 3.8—which supports CUDA 12.8—NVIDIA is extending support to the Blackwell architecture, enabling developers to harness next-generation Tensor Cores with support for all new data types. This includes new narrow precision MX formats and the NVIDIA-developed FP4 format, which increase compute throughput. Figure 1 shows CUTLASS can achieve up to 98% relative peak performance for Tensor Core operations.
Figure 1. Blackwell CUTLASS GEMM performance relative to expected peak, delivering up to 98% of Blackwell peak performance
For DeepSeek-V3 and DeepSeek-R1, grouped GEMMs make up a large portion of the MoE compute required during inference. These operations enable different matrix sizes, scaling factors, and fusions to be grouped and parallelized in a single persistent-kernel launch. With CUTLASS, on Blackwell with FP4, Grouped GEMM kernel performance increases by up to 5x over H200 with FP16.
Figure 2. CUTLASS Grouped GEMM performance for MoE inference used in DeepSeek delivers up to 5x more performance on Blackwell compared to Hopper at various precisions
NVIDIA Nsight Developer Tools
NVIDIA
Nsight Compute
2025.1 is the first official release with support for the Blackwell architecture. Updates include visualization of Blackwell Hardware Tensor Memory in the memory chart as well as Tensor Core performance data.
Figure 3. Tensor Memory traffic in the Nsight Compute memory chart
It also comes with several improvements to the increasingly popular range profiling feature. Users can now collect source-level metrics, including Instructions Executed and memory access information, inside profiled ranges. This update also enables Guided Analysis rules evaluation for ranges. This built-in expertise for identifying performance issues is a key component of NVIDIA Nsight Compute. This release reports kernel stack sizes and adds custom tooltips to help users understand their workload performance.
This release of Compute Sanitizer, an automatic correctness checking tool, adds support for Python call stacks to accurately locate kernel correctness issues when kernels are launched through Python applications. Additionally, new Tensor Core MMA guardrails for Blackwell can report errors related to Tensor Core programming. These are enabled by adding the PTXAS flag
-g-tmem-access-check
when compiling programs. Examples of common errors include access to unallocated tensor memory, invalid addresses, and invalid allocator usage.
Math libraries updates
With CUDA Toolkit 12.8, we have several new library enhancements that leverage the new Blackwell architecture and help accelerate applications in AI, data sciences, graphics and simulation, and high-performance scientific computing.
New features
cuBLAS
APIs
were extended to support microscaled 4-bit and 8-bit floating point mixed-precision tensor core accelerated matrix multiplication for compute capability 10.0 (Blackwell) and higher.
Introduced
initial support for CUDA in Graphics (CIG)
on Windows x64 for NVIDIA Ampere GPU architecture and Blackwell GeForce-class GPUs. CIG contexts are now autodetected, and cuBLAS selects kernels that comply with CIG shared memory usage limits.
cuSOLVER
now supports zsytrf/zsytrs, a complex symmetric direct solver without pivoting.
nvJPEG
now provides support for the Tegra architecture.
NPP
now provides support for the DRIVE Thor architecture.
cudaStreamGetDevice
Applications often use CUDA streams to provide ordered access to GPU resources. An instance of a CUDA stream is associated with a fixed CUDA device. In applications that address multiple devices, there are scenarios where getting a handle to the underlying device for a given stream is useful to tailor the application to device characteristics.
Previously, the CUDA API did not provide a mechanism for retrieving the device associated with a CUDA stream; developers had to track this themselves. The addition of the
cudaStreamGetDevice
CUDA API to retrieve the device associated with a CUDA stream can simplify applications.
Compiler updates
New compiler updates include the following:
The CUDA Toolkit 12.8 release introduces support for
GCC 14
as a host-side compiler.
The default high-level optimizer is now based on
LLVM 18
for the Blackwell architecture.
nvdisasm
now supports emitting JSON formatted SASS disassembly.
Accelerated Python updates
The following two beta releases are now available for Python users:
CUDA Python has released an early prototype of a new idiomatic object model called
cuda.core
and moved the CUDA binding to a submodule,
cuda.bindings
. For more information, see the documentation in the
NVIDIA/cuda-python
GitHub repo.
CUDA Core Compute Libraries (CCCL) has released early prototypes of Python for parallel and cooperative algorithms, enabling you to use thread-level parallelism with user-defined types and functions from pure Python code. Learn more about
CCCL
.
Additionally, the CuPy team is releasing a new version with Blackwell patches validated for general availability.
Feature-complete architectures
With the CUDA Toolkit 12.8 release, we now consider the Maxwell, Pascal, and Volta architectures to be feature-complete and support for them will be frozen in an upcoming release.
This means that, in future releases, no new features will be added to the driver to enable new CUDA Toolkit functionality supporting Maxwell, Pascal, and Volta architectures. End users will be able to run existing software stacks and applications on Maxwell, Pascal, and Volta architectures using the supported upcoming LTS driver branch through its lifecycle.
Starting with release 12.8, developers running offline compilers targeting these architectures will output a warning message when using
nvcc
,
nvrtc
, and
nvjitlink
.
In the next major CUDA Toolkit release, offline compilation support for the Maxwell, Pascal, and Volta architectures will be removed from the compilers. The upcoming LTS driver for production application execution and JIT compilation of Maxwell, Pascal, and Volta applications will be supported for the normal 3-year LTS support window.
For more details, read the
CUDA Toolkit 12.8 Release Notes
.
Summary
The CUDA Toolkit 12.8 release provides full feature support for the NVIDIA Blackwell architecture. This release continues to provide enhanced support for the newest NVIDIA GPUs, accelerated libraries, compilers, and Developer Tools, whether you’re developing applications in C++ or Python.
Want more information? Check out the
CUDA documentation
, browse the latest
NVIDIA Deep Learning Institute (DLI)
offerings, and visit the
NGC catalog
. Ask questions and join the conversation in the
CUDA Developer Forums.
Acknowledgments
Thanks to the following NVIDIA contributors: Stephen Jones, Jackson Marusarz, Becca Zandstein, Andy Terrel, Ashraf Eassa, Matt Nicely, and Mridula Prakash.
CUDA 工具包现已支持 NVIDIA Blackwell 架构
CUDA 工具包
的最新版本 (版本 12.8) 使用最新的 NVIDIA CPU 和 GPU，持续提升数据科学、AI、科学计算以及计算机图形和模拟领域的加速计算性能。本文重点介绍了此版本包含的一些新功能和增强功能：
NVIDIA Blackwell 架构支持
CUDA 图形处理条件节点增强功能
用于大语言模型（LLMs）的 Blackwell CUTLASS 内核
NVIDIA Nsight 开发者工具更新
数学库更新
cudaStreamGetDevice
编译器更新
加速 Python 更新
功能齐全的架构
NVIDIA Blackwell 架构支持
CUDA 工具包 12.8 是该工具包的第一个版本，在整个开发者工具套件 (包括性能工具和分析器、库和编译器) 中支持
NVIDIA Blackwell
架构。Blackwell 由 208 亿个晶体管构建而成，是
NVIDIA Hopper
GPU 中晶体管数量的 2.5 倍以上，是迄今为止最大的 GPU。
Blackwell 支持的主要功能包括：Key Blackwell
采用自定义 Tensor Core 技术的第二代 Transformer 引擎：加速 LLM 和 mixture-of-experts (MoE) 模型的推理和训练。
解压缩：
使用 LZ4、Snappy 和 Deflate 等最新压缩格式，加速数据分析和数据科学工作流的性能。
网络互连：NVLink 和 NVLink Switches 加速万亿参数和数万亿参数 AI 模型的 GPU 间通信性能。
如需详细了解 NVIDIA Blackwell 的领先创新，请参阅 NVIDIA Blackwell 架构技术概览。
使用运行时核选择将 CUDA Graphs 速度提升 2 倍，从而降低延迟推理
借助 Blackwell，CUDA Graphs APIs 仍然是启动 GPU 操作序列重复调用的最高效方式。CUDA Toolkit 12.8 为 CUDA Graphs 引入了更多增强功能，包括其他
条件节点类型
。
在许多应用程序中，对 CUDA Graphs 中工作的执行进行动态控制可以提高图形启动的性能和灵活性。例如，一种算法需要多次迭代一系列运算，直到结果收到某个值以下，现在这种算法无需进行 CPU 控制管理即可完全在 GPU 上运行，从而将开销降低高达 2 倍。CUDA Toolkit 12.8 改进了用于条件图形节点运行时控制的 API。
条件节点包含图形的片段，这些片段可以在图形运行时根据要评估的条件执行或跳过。此类片段可以评估一次 (IF 节点)，也可以在循环中重复评估 (WHILE 节点)。CUDA 12.8 增加了对两种新型条件图形节点的支持：IF/ELSE 组合节点和 SWITCH 节点。
借助 Blackwell 架构，我们改进了 LLM 性能，使包括 DeepSeek-R1 在内的所有推理模型受益。与返回 CPU 进行启动决策相比，CUDA Graphs 增强的 SWITCH 和 IF/ELSE 支持可将运行时内核选择的性能提高 2 倍。
训练：通过减少内核选择对 CPU 的依赖，训练工作负载可维持更多的 GPU Tensor Core 吞吐量，从而提高模型 FLOPS 利用率（MFU）。这提高了使用相同的 GPU 基础架构的性能，减少了训练时间和成本。
推理：对于使用测试时计算的新一代推理模型
，高令牌生成速率至关重要，因为每个推理请求都可以在每个查询中生成大量令牌。CUDA 12.8 新流 API 可减少对主机 CPU 的调用，从而缩短一次内核处理与下一次启动之间的时间，从而提高令牌生成率。这会在固定时间预算内生成更多 token，帮助模型推理更多并提高智能。
如需了解详情，
请参阅使用条件节点的 CUDA 图形中的动态控制流。
适用于 LLMs 的 Blackwell CUTLASS 内核
自 2017 年首次推出以来，
CUTLASS
一直在推动研究人员和开发者在 NVIDIA GPUs 上实施高性能 CUDA 核函数。通过为开发者提供全面的工具来设计自定义操作 (例如 GEMMs 和 Convolutions)，CUTLASS 在开发硬件感知算法方面发挥了至关重要的作用，推动了 FlashAttention 等帮助激发现代 AI 的突破。
随着支持 CUDA 12.8 的 CUTLASS 3.8 的发布，NVIDIA 将扩展对 Blackwell 架构的支持，使开发者能够利用新一代 Tensor Core 来支持所有新的数据类型。这包括新的窄精度 MX 格式和 NVIDIA 开发的 FP4 格式，可提高计算吞吐量。图 1 显示，对于 Tensor Core 运算，CUTLASS 可实现高达 98% 的相对峰值性能。
图 1. Blackwell CUTLASS GEMM 性能相对于预期峰值，可提供高达 98% 的 Blackwell 峰值性能
对于 DeepSeek-V3 和 DeepSeek-R1，分组的 GEMM 在推理期间所需的 MoE 计算中占很大比例。这些运算支持在单个持久性核函数启动中对不同的矩阵大小、缩放系数和融合进行分组和并行化。借助 CUTLASS，在 Blackwell 以 FP4，Grouped GEMM 内核性能增加高达 5 倍，相比使用 FP16 的 H200。
图 2、与 Hopper 相比，DeepSeek 中使用的用于 MoE 推理的 CUTLASS 分组 GEMM 性能在 Blackwell 上在各种精度下的性能提升高达 5 倍
NVIDIA Nsight 开发者工具
NVIDIA
Nsight Compute
2025.1 是首个支持 Blackwell 架构的官方版本。更新包括显存图表中 Blackwell 硬件 Tensor 内存的可视化，以及 Tensor Core 性能数据。
图 3、Nsight Compute 内存图中的 Tensor 内存流量
它还对日益流行的范围分析功能进行了多项改进。用户现在可以在已分析的范围内收集源级指标，包括已执行指令和内存访问信息。此更新还启用了针对范围的引导分析规则评估。这种用于识别性能问题的内置专业知识是 NVIDIA Nsight Compute 的关键组件。此版本报告了内核堆栈大小，并添加了自定义工具提示，以帮助用户了解其工作负载性能。
此版本的 Compute Sanitizer 是一款自动正确性检查工具，增加了对 Python 调用堆栈的支持，可在通过 Python 应用启动内核时准确定位内核正确性问题。此外，用于 Blackwell 的新 Tensor Core MMA 护栏可以报告与 Tensor Core 编程相关的错误。在编译程序时，可以通过添加 PTXAS 标志
-g-tmem-access-check
来启用这些功能。常见错误的示例包括访问未分配的 tensor 内存、无效的地址以及使用无效的分配器。
数学库更新
借助 CUDA 工具包 12.8，我们获得了一些新的增强功能库，这些增强功能利用了新的 Blackwell 架构，并有助于加速 AI、数据科学、图形和仿真以及高性能科学计算领域的应用程序。
新功能
cuBLAS
API
经过扩展，支持微缩 4 位和 8 位浮点混合精度张量核心加速矩阵乘法，可实现 10.0（Blackwell）及更高版本的计算能力。
为 Windows x64 上的 NVIDIA Ampere GPU 架构和 Blackwell GeForce 级 GPU 引入了对 CUDA in Graphics (CIG) 的初步支持。现在，系统会自动检测 CIG 上下文，并且 cuBLAS 会选择符合 CIG 共享内存使用限制的内核。
cuSOLVER
现在支持 zsytrf/zsytrs，这是一款无需旋转的复杂对称直接求解器。
nvJPEG
现在支持 Tegra 架构。
NPP
现在为 DRIVE Thor 架构提供支持。
cudaStreamGetDevice
应用程序通常使用 CUDA 流提供对 GPU 资源的有序访问。CUDA 流实例与固定的 CUDA 设备相关联。在用于处理多台设备的应用中，在某些情况下，为给定流获取底层设备的句柄有助于根据设备特性定制应用。
以前，CUDA API 没有提供检索与 CUDA 流关联的设备的机制；开发者必须自行追踪。添加
cudaStreamGetDevice
CUDA API 以检索与 CUDA 流关联的设备，可以简化应用。
编译器更新
新的编译器更新包括以下内容：
CUDA 工具包 12.8 版本引入了对作为主机端编译器的
GCC 14
的支持。
现在，Blackwell 架构的默认高级优化器基于
LLVM 18
nvdisasm
现在支持发射 JSON 格式的 SASS 反汇编。
加速 Python 更新
以下两个测试版现已面向 Python 用户提供：
CUDA Python 已发布名为
cuda.core
的新惯用对象模型的早期原型，并将 CUDA 绑定移至子模块
cuda.bindings
。有关更多信息，请参阅
NVIDIA/cuda-python
GitHub 存储库中的文档。
CUDA 核心计算库 (
CCCL
) 已发布用于并行和协作算法的早期 Python 原型，使您能够使用线程级并行性以及来自纯 Python 代码的用户定义类型和函数。详细了解 CCCL。
此外，CuPy 团队还将发布新版本，其中的 Blackwell 补丁经过验证，现已全面推出。
功能齐全的架构
在 CUDA 工具包 12.8 版本中，我们现在认为 Maxwell、Pascal 和 Volta 架构功能齐全，并且即将发布的版本将冻结对这些架构的支持。
这意味着，在未来的版本中，不会向驱动添加任何新功能来启用支持 Maxwell、Pascal 和 Volta 架构的新 CUDA 工具包功能。最终用户将能够在其生命周期中使用受支持的即将推出的 LTS 驱动分支，在 Maxwell、Pascal 和 Volta 架构上运行现有的软件堆栈和应用。
从版本 12.8 开始，开发者在运行针对这些架构的离线编译器时，将在使用
nvcc
、
nvrtc
和
nvjitlink
时输出警告消息。
在下一个主要 CUDA 工具包版本中，将从编译器中删除对 Maxwell、Pascal 和 Volta 架构的离线编译支持。即将推出的用于生产应用程序执行的 LTS 驱动以及 Maxwell、Pascal 和 Volta 应用程序的 JIT 编译将在正常的 3 年期 LTS 支持窗口期内获得支持。
如需了解更多详情，请参阅
CUDA Toolkit 12.8 版本说明
。
总结
CUDA 工具包 12.8 版本为 NVIDIA Blackwell 架构提供完整的功能支持。无论您是使用 C++ 还是 Python 开发应用程序，此版本都将继续为最新的 NVIDIA GPU、加速库、编译器和开发者工具提供增强支持。
想要了解更多信息？查看
CUDA 文档
，浏览最新的
NVIDIA Deep Learning Institute (DLI) 产品
，并访问
NGC 目录
。在
CUDA Developer Forums
中提出问题并加入对话。
致谢
感谢以下 NVIDIA 贡献者：Stephen Jones、Jackson Marusarz、Becca Zandstein、Andy Terrel、Ashraf Eassa、Matt Nicely 和 Mridula Prakash。