Advancing Robot Learning, Perception, and Manipulation with Latest NVIDIA Isaac Release
At CES 2025, NVIDIA announced key updates to
NVIDIA Isaac
, a platform of accelerated libraries, application frameworks, and AI models that accelerate the development of AI robots.
NVIDIA Isaac streamlines the development of robotic systems from simulation to real-world deployment. In this post, we discuss all the new advances in NVIDIA Isaac:
Isaac Sim
Isaac Lab
Isaac Manipulator
Isaac Perceptor
What’s new in Isaac Sim 4.5
NVIDIA Isaac Sim
is a reference application built on
NVIDIA Omniverse
that enables you to develop, simulate, and test AI-driven robots in physically based virtual environments.
The
new Isaac Sim 4.5
will offer a number of significant changes, including the following:
A reference application template
Improved URDF import and setup
Improved physics simulation and modeling
New joint visualization tool
Simulation accuracy and statistics
NVIDIA Cosmos world foundation model
Reference application template
Isaac Sim has been redesigned as a customizable reference application. A minimal template for faster startup and a full template with complete functionality along with all dependencies. This enables you to tailor the application to your specific needs, whether for headless applications or full Isaac Sim experiences.
Improved URDF import and setup
Significant improvements have been made to the URDF importer. The User Interface has been simplified to provide a more streamlined process and align with how other formats are imported. You can now individually configure joint drives, making the robot ready to use immediately after import. To assist with joint drive configuration, a natural frequency-based tuning option is provided.
Improved physics simulation and modeling
Isaac Sim 4.5 features significant advancements in physics modeling and simulation. You can define and configure various joint types between robot components, setting parameters like stiffness and damping to fine-tune joint behavior.
New joint visualization tool
A new joint visualization tool enables you to inspect the physics properties of selected prims, including their position, rotation, linear and angular velocities, and accelerations. You can now review and optimize various scene parameters such as deformable surfaces or memory usage, before running simulations.
Figure 1. Various attributes of selected objects and their association with the other objects
Simulation accuracy and statistics
Simulation accuracy is greatly improved with a new implementation of full momentum conservation for rigid bodies and articulations.
You can now also visualize the simulation statistics of objects and scenes that can either interact with each other or are completely independent. You can review different parameters from deformable surfaces to overall memory used in the buffer. This provides a way to troubleshoot and optimize your scene prior to running the simulation.
Figure 2. Simulation statistics associated inside a given scene
NVIDIA Cosmos world foundation model
Also announced at CES, the
NVIDIA Cosmos
world foundation model platform can be used to generate massive amounts of controllable
synthetic data
to train perception robots when paired with Isaac Sim.
In Isaac Sim, you compose
SimReady
3D scenes by unifying diverse data inputs including CAD, lidar-to-point cloud scans, and generated 3D objects from AI models such as
Edify 3D
. Then, you compose and stage the scenario to reflect the specific task the robot must perform, and render images or videos.
Cosmos can ingest images and videos and output photoreal video clips to then retrain policy models.
Video 1. NVIDIA Cosmos: A World Foundation Model Platform for Physical AI
What’s new in Isaac Lab 2.0
NVIDIA Isaac Lab
is an open-source unified framework for
robot learning
to train robot policies. Isaac Lab is built on top of
NVIDIA Isaac Sim
, helping developers and researchers more efficiently build intelligent, adaptable robots with robust, perception-enabled, simulation-trained policies.
A
new 2.0 version of Isaac Lab
includes performance and usability improvements:
Tiled rendering:
Up to a 1.2x boost in tiled rendering speed, which combines outputs from simultaneous simulations into a single, large image rather than processing numerous smaller images from individual cameras.
Quality of life improvements:
Simplified installation process using Python package managers. Isaac Lab will also be available as a container, enabling the movement of workloads across systems without underlying dependencies.
Previewed at CES, humanoid robot developers can also take advantage of the
NVIDIA Isaac GR00T Blueprint
for building custom data pipelines for generating vast amounts of synthetic trajectory data from just a small number of human demonstrations. The GR00T blueprint is currently in invite-only early access. Join the
NVIDIA Humanoid Developer Program
when it becomes widely available in beta.
Video 2. Streamline Data Collection With NVIDIA Isaac GR00T
What’s new in Isaac Manipulator
NVIDIA Isaac Manipulator
, built on ROS 2, is a collection of NVIDIA CUDA-accelerated libraries, AI models, and reference workflows. It enables you to build AI-enabled robot arms, or
manipulators
, that can perceive, understand, and interact with their environments.
Isaac Manipulator now includes new end-to-end reference workflows for pick-and-place and object-following, enabling you to quickly get started on fundamental industrial robot arm tasks:
Object-following
: Shows the robot gripper’s ability to maintain a consistent position relative to a moving object, while maneuvering around obstacles.
Pick-and-place
: Shows how a robot can pick up an object and release it in a predetermined region while avoiding obstacles (Figure 3).
Figure 3. Perception-driven pick-and-place
These reference workflows are now supported in
Isaac Sim
, enabling rapid testing without the need for physical hardware setup.
Developer support and other enhancements include the following:
Performance improvements to
FoundationPose
Updates to
nvblox
for manipulator use cases
A
tutorial
for robot hand-eye calibration
An
Isaac Sim-based tool
for setting and simulating custom grasps for a gripper and object pair
What’s new in Isaac Perceptor
NVIDIA Isaac Perceptor
, built on ROS 2, is a collection of NVIDIA CUDA-accelerated libraries, AI models, and reference workflows for the development of autonomous mobile robots (AMRs). It enables AMRs to perceive, localize, and operate in unstructured environments such as warehouses or factories.
Isaac Perceptor encompasses CUDA-accelerated libraries such as
nvblox
for 3D scene reconstruction and
cuVSLAM
for stereo-visual-inertial SLAM (simultaneous localization and mapping), which you can integrate into existing AMR workflows.
Isaac Perceptor’s latest updates bring significant improvements to AMR’s environmental awareness and operational efficiency in dynamic settings such as warehouses. Key new features and improvements include:
New end-to-end visual SLAM reference workflow
New examples on running nvblox with multiple cameras for 3D scene reconstruction with
people detection and dynamic scene elements
Improved 3D scene reconstruction by
running Isaac Perceptor on multiple RGB-D camera
These updates significantly improve 3D scene reconstruction, leading to higher accuracy and robustness in 3D scene capture and mapping performance in real-world scenarios and complex, dynamic environments.
Figure 4. 3D scene reconstruction using multiple cameras
NVIDIA ecosystem partners such as Orbbec, LIPS (Realsense), StereoLabs (Zed) offer compatible cameras. Developer support also includes the
Mapping and Localization with Isaac Perceptor
tutorial for offline mapping capabilities with the cuVGL and cuVSLAM libraries using Nova sensors.
Ecosystem adoption
Multiple industry partners have announced integrations of NVIDIA Isaac into their platforms and solutions:
Boston Dynamics is using Isaac Lab
and NVIDIA Jetson AGX Orin to enable simulated policies to be directly deployed for inference, simplifying the deployment process.
To train their GR-1 and GR-2 humanoid robots, the
Fourier team turned to NVIDIA Isaac Gym
(now deprecated) for reinforcement learning. They are currently porting their workflows to NVIDIA Isaac Lab.
Foxglove has developed an
extension in Isaac Sim
that enables the real-time visualization of robotics simulation data directly in Foxglove.
Main Street Autonomy’s
Calibration Anywhere software
automates sensor calibration improving sensor fusion for robotics using Isaac Perceptor.
Miso Robotics
automates kitchen tasks using advanced robotics, Isaac Manipulator, and NVIDIA Isaac ROS, enhancing efficiency, consistency, and customer satisfaction in commercial kitchens.
RGo Robotics
and NVIDIA are transforming mobile robotics with advanced AI and perception technologies using Isaac Perceptor and Isaac ROS.
Scaled Foundations
, an NVIDIA inception member, has developed
General Robot Intelligence Development (GRID)
, an advanced cloud-based platform that accelerates robot AI solution development. GRID seamlessly integrates Isaac Sim and Isaac Lab technologies, offering an end-to-end platform for robotics developers and researchers to train, simulate, and deploy their robotics applications. For more information, see
NVIDIA Isaac Sim on GRID
.
Virtual Incision
is using NVIDIA platforms involving Holoscan, IGX, Sensor Bridge, Isaac Sim. They are exploring Cosmos to train, simulate, and test, and also exploring inference AI features for the next generation of assisted robotic surgery devices.
NVIDIA Inception
member and deep tech startup
Wandelbots
is building custom robot simulations with their operating system Wandelbots NOVA, which is seamlessly integrated with Isaac Sim.
Get started developing your own robotics solutions
Sign up for the
NVIDIA Developer Program
for updates on additional resources and reference architectures to support your development goals.
NVIDIA Cosmos
is a platform that helps you build custom world models for physical AI systems and includes pretrained world foundation models for robotics applications.
NVIDIA Isaac Lab
is an open-source unified framework for robot learning to train robot policies.
NVIDIA Isaac ROS
, built on the open-source
ROS 2 software framework
, is a collection of accelerated computing packages and AI models, bringing NVIDIA-acceleration to ROS developers everywhere.
NVIDIA Isaac Sim
, built on
NVIDIA Omniverse
, lets you build your own
OpenUSD
-based applications to design, simulate, test, and train AI-based robots and machines in a physically based virtual environment.
Stay up-to-date on
LinkedIn
,
Instagram
,
X
, and
Facebook
.  For more information, explore our
documentation
and join the Robotics community on the
developer forums
and
YouTube
channels. Follow our
self-paced training
and webinars (
Isaac ROS
and
Isaac Sim
).
利用最新的 NVIDIA Isaac 版本推进机器人学习、感知和操控技术
在 CES 2025 上，NVIDIA 宣布了对
NVIDIA Isaac
的重要更新。NVIDIA Isaac 是一个由加速库、应用框架和 AI 模型组成的平台，可加速 AI 机器人的开发。
NVIDIA Isaac 简化了从仿真到实际部署的机器人系统开发过程。在本文中，我们将讨论 NVIDIA Isaac 的所有新进展：
Isaac Sim
Isaac Lab
Isaac Manipulator
Isaac Perceptor
Isaac Sim 4.5 的新增功能
NVIDIA Isaac Sim
是一款基于
NVIDIA Omniverse
构建的参考应用，使您能够在基于物理的虚拟环境中开发、模拟和测试 AI 驱动的机器人。
新的 Isaac Sim 4.5 将于 1 月底推出，将带来一系列重大变化，包括：
参考应用模板
改进了 URDF 导入和设置
改进了物理模拟和建模
新的关节可视化工具
模拟准确性和统计数据
NVIDIA Cosmos 世界基础模型
参考应用模板
Isaac Sim 经过重新设计，是一款可定制的参考应用。用于加快启动速度的最小模板和具有完整功能以及所有依赖项的完整模板。这使您能够根据自己的特定需求定制应用，无论是用于无外设应用还是完整的 Isaac Sim 体验。
改进了 URDF 导入和设置
对 URDF 导入器进行了重大改进。用户界面经过简化，可提供更精简的流程，并与其他格式的导入方式保持一致。现在，您可以单独配置关节驱动，使机器人在导入后可以立即使用。为辅助关节驱动配置，我们提供了基于自然频率的调整选项。
改进了物理模拟和建模
Isaac Sim 4.5 在物理建模和仿真方面取得了显著进步。您可以定义和配置机器人组件之间的各种关节类型，通过设置刚度和阻尼等参数来微调关节行为。
新的关节可视化工具
借助新的关节可视化工具，您可以检查选定基元的物理属性，包括其位置、旋转、线性和角速度以及加速度。现在，您可以在运行模拟之前查看和优化各种场景参数，例如可变形表面或显存使用情况。
图 1、所选对象的各种属性及其与其他对象的关联
模拟准确性和统计数据
通过对刚体和关节实施新的全动量守势，模拟准确性得到大幅提高。
现在，您还可以可视化物体和场景的仿真统计信息，这些物体和场景可以相互交互，也可以完全独立。您可以查看从可变形表面到缓冲区中使用的总显存的不同参数。这样可以在运行仿真之前对场景进行故障排除和优化。
图 2、给定场景内关联的模拟统计数据
NVIDIA Cosmos 世界基础模型
此外，在 CES 上宣布的
NVIDIA Cosmos World Foundation
模型平台与 Isaac Sim 搭配使用时，可用于生成大量可控制的合成数据来训练感知机器人。
在 Isaac Sim 中，您可以通过统一不同的数据输入 (包括 CAD、激光雷达到点云扫描) 以及从 AI 模型 (例如
Edify 3D
) 生成的 3D 对象来合成
SimReady 3D 场景
。然后，合成场景并对其进行暂存，以反映机器人必须执行的特定任务，并渲染图像或视频。
Cosmos 可以提取图像和视频，输出逼真的视频片段，然后重新训练策略模型。
视频 1、NVIDIA Cosmos：适用于物理 AI 的世界基础模型平台
Isaac Lab 2.0 的新增功能
NVIDIA Isaac Lab
是一个用于机器人学习的开源统一框架，用于训练机器人策略。Isaac Lab 基于 NVIDIA Isaac Sim 构建，可帮助开发者和研究人员更高效地构建智能、适应性强的机器人，并提供稳健、支持感知、经过仿真训练的策略。
新版 Isaac Lab 2.0 将于本月底推出，其中包括以下性能和可用性改进：
平铺式渲染：
平铺式渲染速度最高可提升 1.2 倍，可将同时模拟的输出结果整合到单个大型图像中，而无需处理来自单个摄像头的大量较小图像。
改善生活质量：
使用 Python 包管理器简化安装流程。Isaac Lab 还可用作容器，无需底层依赖项即可跨系统迁移工作负载。
在 CES 展会上，类人型机器人开发者还可以利用
NVIDIA Isaac GR00T Blueprint
构建自定义数据流水线，通过少量人类演示生成大量合成轨迹数据。GR00T 蓝图目前处于邀请抢先体验阶段。当 NVIDIA Humanoid 开发者计划在测试版中广泛推出时，加入该计划。
视频 2、借助 NVIDIA Isaac GR00T 简化数据采集
Isaac Manipulator 新增功能
NVIDIA Isaac Manipulator
基于 ROS 2 构建，是 NVIDIA CUDA 加速库、AI 模型和参考工作流的集合。它使您能够构建支持 AI 的机械臂 (Manipulator)，能够感知和理解周围环境并与之交互。
Isaac Manipulator 现在包含新的端到端参考工作流，用于拾放和对象跟踪，能够让您快速开始执行基本的工业机器人手臂任务。
物体追踪
：展示机器人抓手在绕过障碍物时相对于移动物体保持稳定位置的能力。
拾放
：展示机器人如何拾取物体并将其释放到预定区域，同时避开障碍物 (Figure 3)。
图 3、感知驱动拾放
这些参考工作流程现在支持在
Isaac Sim
中使用，实现了无需物理硬件设置的快速测试。
开发者支持和其他增强功能包括：
性能提升
FoundationPose
更新
NVBlox
操纵器用例
用于机器人手眼校准的
教程
一款基于 Isaac Sim 的工具
，用于为抓手和对象对设置和模拟自定义抓取
Isaac Perceptor 新增功能
NVIDIA Isaac Perceptor
基于 ROS 2 构建，是 NVIDIA CUDA 加速库、AI 模型和参考工作流的集合，用于开发自主移动机器人 (Autonomous Mobile Robots, AMRs)。它使 AMRs 能够在仓库或工厂等非结构化环境中进行感知、定位和操作。
Isaac Perceptor 包含 CUDA 加速库，例如用于 3D 场景重建的
nvblox
和用于立体视觉惯性 SLAM（同步定位和映射）的
cuVSLAM
，您可以将其集成到现有的自动移动机器人（AMR）工作流程中。
Isaac Perceptor 的最新更新显著提高了 AMR 在仓库等动态环境中的环境意识和运营效率。主要的新功能和改进包括：
新的端到端可视化 SLAM 参考工作流
关于使用多个摄像头运行 NVIDIA 的 nvblox 的新示例，用于 3D 场景重建的
人员检测和动态场景元素
通过
在多个 RGB-D 摄像头上运行 Isaac Perceptor
，改进了 3D 场景重建
这些更新显著改进了 3D 场景重建，从而提高了真实场景和复杂动态环境中 3D 场景捕获和映射性能的准确性和可靠性。
图 4、使用多个摄像头重建 3D 场景
Orbbec、LIPS (Realsense)、StereoLabs (Zed) 等 NVIDIA 生态系统合作伙伴提供兼容的摄像头。开发者支持还包括“Mapping and Localization with Isaac Perceptor”(
使用 Isaac Perceptor 进行映射和定位
) 教程，该教程通过使用 Nova 传感器的 cuVGL 和 cuVSLAM 库提供离线映射功能。
采用生态系统
多个行业合作伙伴已宣布将 NVIDIA Isaac 集成到其平台和解决方案中：
Boston Dynamics 正在使用 Isaac Lab
和 NVIDIA Jetson AGX Orin 直接部署模拟策略以进行推理，从而简化部署流程。
为了训练他们的 GR-1 和 GR-2 类人型机器人，
Fourier 团队转而使用 NVIDIA Isaac Gym
(现已弃用) 进行强化学习。他们目前正在将工作流移植到 NVIDIA Isaac Lab。
Foxglove
在 Isaac Sim 中开发了一个扩展程序
，可直接在 Foxglove 中实时可视化机器人仿真数据。
Main Street Autonomy 的
Calibration Anywhere 软件
使用 Isaac Perceptor 自动进行传感器校准，改善机器人的传感器融合。
Miso Robotics 使用先进的机器人、
Isaac Manipulator 和 NVIDIA Isaac ROS 实现厨房任务自动化，提高商业厨房的效率、一致性和客户满意度。
RGo Robotics 和 NVIDIA 正在利用 Isaac Perceptor 和 Isaac ROS，利用先进的 AI 和感知技术变革移动机器人技术
。
作为 NVIDIA 初创加速计划的成员，
Scaled Foundations
开发了通用机器人智能开发 (
GRID
)，这是一个基于云的先进平台，可加速机器人 AI 解决方案的开发。GRID 无缝集成了 NVIDIA Isaac Sim 和 NVIDIA Isaac Lab 技术，为机器人开发者和研究人员提供了一个端到端平台，用于训练、模拟和部署其机器人应用。有关更多信息，请参阅基于 GRID 的 NVIDIA Isaac Sim。
Virtual Incision
正在使用 NVIDIA 平台，涉及 Holoscan、IGX、Sensor Bridge、Isaac Sim。他们正在探索 Cosmos 进行训练、模拟和测试，还在探索用于新一代机器人辅助手术设备的推理 AI 功能。
NVIDIA Inception 计划会员兼深度技术初创公司
Wandelbots
正在使用其操作系统 Wandelbots NOVA 构建自定义机器人仿真，该系统与 Isaac Sim 无缝集成。
开始开发您自己的机器人开发解决方案
注册
NVIDIA 开发者计划
，获取有关其他资源和参考架构的更新信息，以支持您的开发目标。
NVIDIA Cosmos 平台
可帮助您为物理 AI 系统构建自定义世界模型，并包含用于机器人应用的预训练世界基础模型。
NVIDIA Isaac Lab 是一个用于机器人学习的开源统一框架
，用于训练机器人策略。
NVIDIA Isaac ROS
基于开源
ROS 2
软件框架构建，是加速计算包和 AI 模型的集合，可为各地的 ROS 开发者提供 NVIDIA 加速。
NVIDIA Isaac Sim
基于
NVIDIA Omniverse
构建，可让您构建自己的基于
OpenUSD
的应用，以便在基于物理的虚拟环境中设计、仿真、测试和训练基于 AI 的机器人和机器。
关注
LinkedIn
、
Instagram
、
X
和
Facebook
，了解最新动态。如需了解更多信息，请浏览我们的文档，并在开发者论坛和
YouTube
频道上加入机器人开发社区。关注我们的自主培训和网络会议 (Isaac ROS 和 Isaac Sim)。