Over the past year,
generative AI
has transformed the way people live, work and play, enhancing everything from writing and content creation to gaming, learning and productivity. PC enthusiasts and developers are leading the charge in pushing the boundaries of this groundbreaking technology.
Countless times, industry-defining technological breakthroughs have been invented in one place — a garage. This week marks the start of the
RTX AI Garage
series, which will offer routine content for developers and enthusiasts looking to learn more about NVIDIA NIM microservices and AI Blueprints, and how to build AI agents, creative workflow, digital human, productivity apps and more on AI PCs. Welcome to the
RTX AI Garage
.
This first installment spotlights announcements made earlier this week at
CES
, including
new AI foundation models
available on
NVIDIA RTX AI PCs
that take digital humans, content creation, productivity and development to the next level.
These models — offered as
NVIDIA NIM
microservices — are powered by new
GeForce RTX 50 Series GPUs
. Built on the NVIDIA Blackwell architecture, RTX 50 Series GPUs deliver up to 3,352 trillion AI operations per second of performance, 32GB of VRAM and feature FP4 compute, doubling AI inference performance and enabling generative AI to run locally with a smaller memory footprint.
NVIDIA also introduced
NVIDIA AI Blueprints
— ready-to-use, preconfigured workflows, built on NIM microservices, for applications like digital humans and content creation.
NIM microservices and AI Blueprints empower enthusiasts and developers to build, iterate and deliver AI-powered experiences to the PC faster than ever. The result is a new wave of compelling, practical capabilities for PC users.
Fast-Track AI With NVIDIA NIM
There are two key challenges to bringing AI advancements to PCs. First, the pace of AI research is breakneck, with new models appearing daily on platforms like Hugging Face, which now hosts over a million models. As a result, breakthroughs quickly become outdated.
Second, adapting these models for PC use is a complex, resource-intensive process. Optimizing them for PC hardware, integrating them with AI software and connecting them to applications requires significant engineering effort.
NVIDIA NIM helps address these challenges by offering prepackaged, state-of-the-art AI models optimized for PCs. These NIM microservices span model domains, can be installed with a single click, feature application programming interfaces (APIs) for easy integration, and harness NVIDIA AI software and RTX GPUs for accelerated performance.
At CES, NVIDIA announced a pipeline of NIM microservices for RTX AI PCs, supporting use cases spanning large language models (LLMs), vision-language models, image generation, speech, retrieval-augmented generation (RAG), PDF extraction and computer vision.
The new
Llama Nemotron
family of open models provide high accuracy on a wide range of agentic tasks. The Llama Nemotron Nano model, which will be offered as a NIM microservice for RTX AI PCs and workstations, excels at agentic AI tasks like instruction following, function calling, chat, coding and math.
Soon, developers will be able to
quickly download
and run these microservices on Windows 11 PCs using Windows Subsystem for Linux (WSL).
To demonstrate how enthusiasts and developers can use NIM to build AI agents and assistants, NVIDIA previewed Project R2X, a vision-enabled PC avatar that can put information at a user’s fingertips, assist with desktop apps and video conference calls, read and summarize documents, and more.
Sign up
for Project R2X updates.
By using NIM microservices, AI enthusiasts can skip the complexities of model curation, optimization and backend integration and focus on creating and innovating with cutting-edge AI models.
What’s in an API?
An API is the way in which an application communicates with a software library. An API defines a set of “calls” that the application can make to the library and what the application can expect in return. Traditional AI APIs require a lot of setup and configuration, making AI capabilities harder to use and hampering innovation.
NIM microservices expose easy-to-use, intuitive APIs that an application can simply send requests to and get a response. In addition, they’re designed around the input and output media for different model types. For example, LLMs take text as input and produce text as output, image generators convert text to image, speech recognizers turn speech to text and so on.
The microservices are designed to integrate seamlessly with leading AI development and agent frameworks such as AI Toolkit for VSCode, AnythingLLM, ComfyUI, Flowise AI, LangChain, Langflow and LM Studio. Developers can easily download and deploy them from
build.nvidia.com
.
By bringing these APIs to RTX, NVIDIA NIM will accelerate AI innovation on PCs.
Enthusiasts are expected to be able to experience a range of NIM microservices using an upcoming release of the
NVIDIA ChatRTX
tech demo.
A Blueprint for Innovation
By using state-of-the-art models, prepackaged and optimized for PCs, developers and enthusiasts can quickly create AI-powered projects. Taking things a step further, they can combine multiple AI models and other functionality to build complex applications like digital humans, podcast generators and application assistants.
NVIDIA AI Blueprints, built on NIM microservices, are reference implementations for complex AI workflows. They help developers connect several components, including libraries, software development kits and AI models, together in a single application.
AI Blueprints include everything that a developer needs to build, run, customize and extend the reference workflow, which includes the reference application and source code, sample data, and documentation for customization and orchestration of the different components.
At CES, NVIDIA announced two AI Blueprints for RTX: one for PDF to podcast, which lets users generate a podcast from any PDF, and another for 3D-guided generative AI, which is based on
FLUX.1 [dev]
and expected be offered as a NIM microservice, offers artists greater control over text-based image generation.
With AI Blueprints, developers can quickly go from AI experimentation to AI development for cutting-edge workflows on RTX PCs and workstations.
Built for Generative AI
The new GeForce RTX 50 Series GPUs are purpose-built to tackle complex generative AI challenges, featuring fifth-generation Tensor Cores with FP4 support, faster G7 memory and an AI-management processor for efficient multitasking between AI and creative workflows.
The GeForce RTX 50 Series adds FP4 support to help bring better performance and more models to PCs. FP4 is a lower quantization method, similar to file compression, that decreases model sizes. Compared with FP16 — the default method that most models feature — FP4 uses less than half of the memory, and 50 Series GPUs provide over 2x performance compared with the previous generation. This can be done with virtually no loss in quality with advanced quantization methods offered by
NVIDIA TensorRT Model Optimizer
.
For example, Black Forest Labs’ FLUX.1 [dev] model at FP16 requires over 23GB of VRAM, meaning it can only be supported by the GeForce RTX 4090 and professional GPUs. With FP4, FLUX.1 [dev] requires less than 10GB, so it can run locally on more GeForce RTX GPUs.
With a GeForce RTX 4090 with FP16, the FLUX.1 [dev] model can generate images in 15 seconds with 30 steps. With a GeForce RTX 5090 with FP4, images can be generated in just over five seconds.
Get Started With the New AI APIs for PCs
NVIDIA NIM microservices and AI Blueprints are expected to be available starting next month, with initial hardware support for GeForce RTX 50 Series, GeForce RTX 4090 and 4080, and NVIDIA RTX 6000 and 5000 professional GPUs. Additional GPUs will be supported in the future.
NIM-ready RTX AI PCs are expected to be available from Acer, ASUS, Dell, GIGABYTE, HP, Lenovo, MSI, Razer and Samsung, and from local system builders Corsair, Falcon Northwest, LDLC, Maingear, Mifcon, Origin PC, PCS and Scan.
GeForce RTX 50 Series GPUs and laptops deliver game-changing performance, power transformative AI experiences, and enable creators to complete workflows in record time. Rewatch NVIDIA CEO Jensen Huang’s
keynote
to learn more about NVIDIA’s AI news unveiled at CES.
See
notice
regarding software product information.
Categories:
Generative AI
Tags:
AI Decoded
|
Artificial Intelligence
|
GeForce
|
NVIDIA RTX
|
RTX AI Garage
過去一年來，
生成式AI
改變了人們的生活、工作和娛樂方式，從寫作、內容創作，到遊戲、學習和生產等領域，都有進一步的提升。PC 愛好者與開發人員正在率先推動這項開創性技術的發展。
此外，帶領產業變革的技術突破，有無數次都是在「車庫」這個地方發明出來的。從本週開始，我們將推出
RTX AI Garage
系列，定期針對開發人員和愛好者發布內容，幫助他們深入瞭解 NVIDIA NIM 微服務與AI Blueprints，以及如何在 AI PC 上建構AI代理、創意工作流程、數位人、生產力應用程式等。歡迎來到
RTX AI Garage
。
首期我們將聚焦本週稍早在
CES
上發表的內容，包括在
NVIDIA RTX AI PC
上可使用的
全新AI基礎模型
，這些模型會將數位人、內容創作、生產力和開發提升至下一個境界。
這些模型將以
NVIDIA NIM
微服務形式提供，由全新
GeForce RTX 50 系列 GPU
驅動。RTX 50 系列 GPU 建立在 NVIDIA Blackwell 架構上，可提供高達每秒 3,352 兆次的AI運算效能、32GB 的 VRAM 與 FP4 運算，讓AI推論效能提升一倍，使生成式AI能以更少的記憶體用量在本機上執行。
NVIDIA 同時推出了
NVIDIA AI Blueprints
，這些可立即使用和預先設置的工作流程為 NIM 微服務的一環，適用於數位人類與內容創作等應用程式。
NIM 微服務與AI Blueprints能讓愛好者與開發人員以前所未有的速度快速為PC 建構、迭代和提供人AI驅動的各式體驗。為 PC 使用者帶來新一波強大的實用功能。
利用 NVIDIA NIM 快速建立AI途徑
將AI進展帶入 PC 存在兩項主要挑戰。 首先，AI研究的步伐正在飛速邁進，每天都有新的模型出現在如 Hugging Face 等平台上，該平台目前已有超過一百萬組模型。因此，技術突破很快就會過時。
其次，將這些模型調適為 PC 所用，是一項複雜且需多方資源配合的過程。將這些模型針對 PC 硬體進行最佳化，並與AI軟體整合，再連接至應用程式，需要大量的工程工作。
NVIDIA NIM 針對 PC 提供了預先封裝、最先進和最佳化的AI模型，協助解決這些挑戰。這些 NIM 微服務涵蓋各模型領域，按一下即可安裝，藉由應用程式設計介面 (APIs) 輕鬆整合，並利用 NVIDIA AI軟體與 RTX GPU 加速效能。
在 CES 上，NVIDIA 宣布為 RTX AI PC 提供一系列 NIM 微服務工作流程，支援大型語言模型 (LLM)、
視覺語言模型、影像生成功能、語音、檢索增強生成功能 (RAG)、PDF 擷取與電腦視覺等應用場景。
全新的
Llama Nemotron
系列開放式模型，在各種代理式任務中，提供了極高的精準度。Llama Nemotron Nano 模型將作為 NIM 微服務提供給 RTX AI PC 與工作站，在指令追蹤、功能呼叫、聊天、
編碼與運算等代理式AI任務中表現出眾。
很快地，開發人員將能夠在使用 Windows Subsystem for Linux (WSL) 的 Windows 11 PC 上
快速下載
並執行這些微服務。
為了向愛好者與開發人員展示如何使用 NIM 來建構AI代理與助理，NVIDIA 公布了 Project R2X 預覽，
這是一種具有視覺能力的 PC 數位化身，它可以讓使用者所需的資訊觸手可及、協助使用桌面應用程式、進行視訊會議通話、閲讀與總結文件等功能。
訂閱
以獲取 Project R2X 的最新消息。
透過 NIM 微服務的使用，AI愛好者可跳過模型策劃、最佳化與後端整合等繁複過程，專注在尖端AI模型的創造與創新。
API 中有哪些內容？
API 是應用程式與軟體庫溝通的方式。API 定義了一組「呼叫」，應用程式可呼叫軟體庫，以及可獲得的預期回應。傳統的AI API 需要大量的設定與配置，使得AI功能的使用難度更高，並阻礙創新。
NIM 微服務提供易於使用、直覺式 API，讓應用程式輕鬆傳送要求並獲得回應。此外，這些微服務都是依照不同類型的模型及其輸入與輸出的介質而設計的。例如，LLM 會輸入文字，然後輸出文字；影像生成器會將文字轉換為影像；語音辨識器會將語音轉換為文字等。
這些微服務的設計，將讓先進的AI開發與代理框架無縫整合，例如 AI Toolkit for VSCode、AnythingLLM、ComfyUI、Flowise AI、LangChain、Langflow 與 LM Studio 等。開發人員可前往
build.nvidia.com
輕鬆下載並部署。
透過將這些 API 引入 RTX，NVIDIA NIM 將加速 PC 的AI創新。
愛好將可藉由即將發布的
NVIDIA ChatRTX
技術示範影片，體驗一系列的 NIM 微服務。
為創新而生的藍圖
開發人員與愛好者可透過針對 PC 打造的預先封裝、最佳化和最先進的模型，快速建立由AI驅動的專案。
之後更能進一步結合多種AI模型和其他功能，建立複雜的應用程式，例如數位人類、Podcast 生成器與應用程式助理。
NVIDIA AI Blueprints以 NIM 微服務為基礎，為更複雜AI工作流程提供了參考實例。它們可協助開發人員將多個元件，包括函式庫、軟體開發套件與AI模型，連結至單一的應用程式中。
AI Blueprints包含了開發人員在建立、執行、自訂及擴展參考工作流程時所需的一切內容，其中包括參考應用程式與原始程式碼、樣本資料，以及自訂與協調不同元件的文件。
在 CES 上，NVIDIA 宣布為 RTX 提供兩種AI Blueprints：一種是由 PDF 生成 Podcast，使用者可從任何 PDF 生成 Podcast；另一種是由 3D 引導的生成式AI，以
FLUX.1 [dev]
為基礎，預計將作為 NIM 微服務提供，讓藝術家更能掌控以文字生成的影像功能。
開發人員可利用AI Blueprints，快速地將AI從實驗階段帶入開發階段，在 RTX PC 與工作站上建立尖端工作流程。
專為生成式AI而打造
全新 GeForce RTX 50 系列 GPU 專為解決複雜的生成式AI挑戰而打造，採用 FP4 支援的第五代 Tensor
核心、速度更快的 G7 記憶體與AI管理處理器，能高效地在AI與創作工作流程之間同時執行多項任務。
GeForce RTX 50 系列新增了 FP4 的支援，創造出適用於 PC 的出色效能及更多模型。FP4 是一種輕量化方法，類似於檔案壓縮，可縮減模型大小。與 FP16 (大多數模型採用的預設方式) 相比，FP4 使用的記憶體不到一半，而 50 系列 GPU 提供的效能較前一代高出 2 倍之多。
NVIDIA TensorRT Model Optimizer
提供先進的量化方法，幾乎不會對品質造成損失。
例如，在 FP16 中的 Black Forest Lab 的 FLUX.1 [dev] 模型需要超過 23GB 的 VRAM，也就是說，只有 GeForce RTX 4090 與專業 GPU 才能支援。若使用了 FP4，FLUX.1 [dev] 只需要不到 10GB，因此只要本機有足夠的 GeForce RTX GPU 即可執行。
採用 FP16 的 GeForce RTX 4090，FLUX.1 [dev] 模型需經過 30 個步驟在 15 秒內生成影像。而採用 FP4 的 GeForce RTX 5090，可在五秒多一點的時間內生成影像。
立即開始使用針對 PC 設計的全新AI API
NVIDIA NIM 微服務與 AI Blueprints預計將於下個月開始提供。初期將由 GeForce RTX 50 系列、GeForce RTX 4090 和 4080 與 NVIDIA RTX 6000 與 5000 專業 GPU 提供硬體支援。未來將會支援更多的 GPU。
支援 NIM 的 RTX AI PC 預計將由 Acer、ASUS、Dell、GIGABYTE、HP、Lenovo、MSI、Razer 和 Samsung ，以及 Corsair、Falcon Northwest、LDLC、Maingear、Mifcon、Origin PCS 和 Sca等系統製造商 提供。
GeForce RTX 50 系列 GPU 與筆記型電腦提供創新效能，為AI帶來變革性的體驗，讓創作者可在前所未有的時限內完成工作流程。請繼續觀看 NVIDIA 執行長黃仁勳的
專題演講
，深入瞭解 NVIDIA 在 CES 上發表的AI最新消息。
請參閱更多軟體產品資訊
通知
。
Categories:
生成式人工智慧
Tags:
AI Decoded
|
Artificial Intelligence
|
GeForce
|
NVIDIA RTX
|
RTX AI Garage