Three Building Blocks for Creating AI Virtual Assistants for Customer Service with an NVIDIA AI Blueprint
In today’s fast-paced business environment, providing exceptional customer service is no longer just a nice-to-have—it’s a necessity. Whether addressing technical issues, resolving billing questions, or providing service updates, customers expect quick, accurate, and personalized responses at their convenience. However, achieving this level of service comes with significant challenges.
Legacy approaches, such as static scripts or manual processes, often fall short when it comes to delivering personalized and real-time support. Additionally, many customer service operations rely on sensitive and fragmented data, which is subject to strict data governance and privacy regulations. With the rise of generative AI, companies aim to revolutionize customer service by enhancing operational efficiency, cutting costs, and maximizing ROI.
Integrating AI into existing systems presents challenges related to transparency, accuracy, and security, which can impede adoption and disrupt workflows. To overcome these hurdles, companies are leveraging generative AI-powered virtual assistants to manage a wide range of tasks, ultimately improving response times and freeing up resources.
This post outlines how developers can use the
NVIDIA AI Blueprint for AI virtual assistants
to scale operations with generative AI. By leveraging this information, including sample code, businesses can meet the growing demands for exceptional customer service while ensuring data integrity and governance. Whether improving existing systems or creating new ones, this blueprint empowers teams to meet customer needs with efficient and meaningful interactions.
Smarter AI virtual assistants with an AI query engine using retrieval-augmented generation
When building an AI virtual assistant, it’s important to align with the unique use case requirements, institutional knowledge, and needs of the organization. Traditional bots, however, often rely on rigid frameworks and outdated methods that struggle to meet the evolving demands of today’s customer service landscape.
Across every industry, AI-based assistants can be transformational. For example, telecommunications companies, and the majority of retail and service providers, can use AI virtual assistants to enhance customer experience by offering support 24 hours a day, 7 days a week while handling a wide range of customer queries in multiple languages and providing dynamic, personalized interactions that streamline troubleshooting and account management. This helps reduce wait times and ensures consistent service across diverse customer needs.
Another example is within the healthcare insurance payor industry, where ensuring a positive member experience is critical. Virtual assistants enhance this experience by providing personalized support to members, addressing their claims, coverage inquiries, benefits, and payment issues, all while ensuring compliance with healthcare regulations. This also helps reduce the administrative burden on healthcare workers.
With the NVIDIA AI platform, organizations can create an AI query engine that uses
retrieval-augmented generation (RAG)
to connect AI applications to enterprise data. The AI virtual assistant blueprint enables developers to quickly get started building solutions that provide enhanced customer experiences. It is built using the following
NVIDIA NIM
microservices:
NVIDIA NIM for LLM:
Brings the power of state-of-the-art large language models (LLMs) to applications, providing unmatched natural language processing with remarkable efficiency.
Llama 3.1 70B Instruct NIM
:
Powers complex conversations with superior contextual understanding, reasoning, and text generation.
NVIDIA NeMo
Retriever NIM:
This collection provides easy access to state-of-the-art models that serve as foundational building blocks for RAG pipelines. These pipelines, when integrated into virtual assistant solutions, enable seamless access to enterprise data, unlocking institutional knowledge via fast, accurate, and scalable answers.
NeMo
Retriever Embedding NIM
:
Boosts text question-answering retrieval performance, providing high-quality embeddings for the downstream virtual assistant.
NeMo
Retriever Reranking NIM
:
Enhances the retrieval performance further with a fine-tuned reranker, finding the most relevant passages to provide as context when querying an LLM.
The blueprint is designed to integrate seamlessly with existing customer service applications without breaking information security mandates. Thanks to the portability of NVIDIA NIM, organizations can integrate data wherever it resides. By bringing generative AI to the data, this architecture enables AI virtual assistants to provide more personalized experiences tailored to each customer by leveraging their unique profiles, user interaction histories, and other relevant data.
A blueprint is a starting point that can be customized for an enterprise’s unique use case.  For example, integrate other NIM microservices, such as the
Nemotron 4 Hindi 4B Instruct
, to enable an AI virtual assistant to communicate in the local language. Other microservices can enable additional capabilities such as synthetic data generation and model fine-tuning to better align with your specific use case requirements. Give the AI virtual assistant a humanlike interface when connected to the digital human AI Blueprint.
With the implementation of a RAG backend with proprietary data (both company and user profile and their specific data), the AI virtual assistant can engage in highly contextual conversations, addressing the specifics of each customer’s needs in real-time. Additionally, the solution operates securely within your existing governance frameworks, ensuring compliance with privacy and security protocols especially when working with sensitive data.
Three building blocks for creating your own AI virtual assistant
As a developer, you can build your own AI virtual assistant that retrieves the most relevant and up-to-date information, in real time, with ever-improving humanlike responses. Figure 1 shows the AI virtual assistant architecture diagram which includes three functional components.
Figure 1. The NVIDIA AI Blueprint for AI virtual assistants
1. Data ingestion and retrieval pipeline
Pipeline administrators use the ingestion pipeline to load structured and unstructured data into the databases. Examples of structured data include customer profiles, order history, and order status. Unstructured data includes product manuals, the product catalog, and supporting material such as FAQ documents.
2. AI agent
The AI virtual assistant is the second functional component. Users interact with the virtual assistant through a user interface. An AI agent, implemented in the LangGraph agentic LLM programming framework, plans how to handle complex customer queries and solves recursively. The LangGraph agent uses the tool calling feature of the
Llama 3.1 70B Instruct NIM
to retrieve information from both the unstructured and structured data sources, then generates an accurate response.
The AI agent also uses short-term and long-term memory functions to enable multi-turn conversation history. The active conversation queries and responses are embedded so they can be retrieved later in the conversation as additional context. This allows more human-like interactions and eliminates the need for customers to repeat information they’ve already shared with the agent.
Finally, at the end of the conversation, the AI agent summarizes the discussion along with a sentiment determination and stores the conversation history in the structured database. Subsequent interactions from the same user can be retrieved as additional context in future conversations. Call summarization and conversation history retrieval can reduce call time and improve customer experience. Sentiment determination can provide valuable insights to the customer service administrator regarding the agent’s effectiveness.
3. Operations pipeline
The customer operations pipeline is the third functional component of the overall solution. This pipeline provides important information and insight to the customer service operators. Administrators can use the operations pipeline to review chat history, user feedback, sentiment analysis data, and call summaries. The analytics microservice, which leverages the Llama 3.1 70B Instruct NIM, can be used to generate analytics such as average call time, time to resolution, and customer satisfaction. The analytics are also leveraged as user feedback to retrain the LLM models to improve accuracy.
You can find the complete example of how to get started with this Blueprint on the
NVIDIA AI Blueprint GitHub repository.
Get to production with NVIDIA partners
NVIDIA consulting partners are helping enterprises adopt world-class AI virtual assistants built using NVIDIA accelerated computing and
NVIDIA AI Enterprise software
, which includes NeMo, NIM microservices, and AI Blueprints.
Accenture
The Accenture AI Refinery
built on
NVIDIA AI Foundry
helps design autonomous, intent-driven customer interactions, enabling businesses to tailor the journey to the individual through innovative channels such as digital humans or interaction agents. Specific use cases can be tailored to meet the needs of each industry, for example, telco call centers, insurance policy advisors, pharmaceutical interactive agents or automotive dealer network agents.
Deloitte
Deloitte Frontline AI enhances the customer service experience with digital avatars and LLM agents built with NVIDIA AI Blueprints that are accelerated by NVIDIA technologies such as NVIDIA ACE, NVIDIA Omniverse, NVIDIA Riva, and NIM.
Wipro
Wipro Enterprise Generative AI (WeGA) Studio accelerates industry-specific use cases including contact center agents across healthcare, financial services, retail, and more.
Tech Mahindra
Tech Mahindra is leveraging the NVIDIA AI Blueprint for digital humans to build solutions for customer service. Using RAG and NVIDIA NeMo, the solution provides the ability for a trainee to stop an agent during a conversation by raising a hand to ask clarifying questions. The system is designed to connect with microservices on the backend with a refined learning management system) which can be deployed across many industry use cases.
Infosys
Infosys Cortex
, part of
Infosys Topaz
, is an AI-driven customer engagement platform that integrates NVIDIA AI Blueprints and the NVIDIA NeMo, Riva, and ACE technologies for generative AI, speech AI, and digital human capabilities to deliver specialized and individualized, proactive, and on-demand assistance to every member of a customer service organization, consequently playing a pivotal role in enhancing customer experience, improving operational efficiency, and reducing costs.
Tata Consultancy Services
The Tata Consultancy Services (TCS) virtual agent, powered by NVIDIA NIM, and integrated with ServiceNow’s IT Virtual Agent is designed to optimize IT and HR support. This solution uses prompt-tuning and RAG to improve response times, accuracy, and provide multi-turn conversational capabilities. Benefits include reduced service desk costs, fewer support tickets, enhanced knowledge utilization, faster deployment, and a better overall employee and customer experience.
Quantiphi
Quantiphi
is integrating NVIDIA AI Blueprints into its conversational AI solutions to enhance customer service with lifelike digital avatars. These state-of-the-art avatars, powered by NVIDIA Tokkio and ACE technologies,
NVIDIA NIM microservices
and
NVIDIA NeMo
, seamlessly integrate with existing enterprise applications, enhancing operations and customer experiences with increased realism. Fine-tuned NIM deployments for digital avatar workflows have proven to be highly cost-effective, reducing enterprise spending on tokens.
SoftServe
SoftServe Digital Concierge
, accelerated by NVIDIA AI Blueprints and NVIDIA NIM microservices, uses NVIDIA ACE, NVIDIA Riva, and the NVIDIA Audio2Face NIM microservice to deliver a highly realistic virtual assistant. Thanks to the Character Creator tool, it delivers speech and facial expressions with remarkable accuracy and lifelike detail.
With RAG capabilities from NVIDIA NeMo Retriever, SoftServe Digital Concierge can intelligently respond to customer queries by referencing context and delivering specific, up-to-date information. It simplifies complex queries into clear, concise answers and can also provide detailed explanations when needed.
EXL
EXL’s Smart Agent Assist offering is a contact center AI solution leveraging NVIDIA Riva, NVIDIA NeMo, and NVIDIA NIM microservices. EXL plans to augment their solution using the NVIDIA AI Blueprint for AI virtual agents.
This week at
NVIDIA AI Summit India
, NVIDIA consulting partners announced a collaboration with NVIDIA to transform India into a Front Office for AI. Using NVIDIA technologies, these consulting giants can help customers tailor the customer service agent blueprint to build unique virtual assistants using their preferred AI model—including sovereign LLMs from India-based model makers—and run it in production efficiently on the infrastructure of their choice.
Get started
To try the blueprint for free, and to see system requirements, navigate to the
Blueprint Card
.
To start building applications using those microservices, visit the
NVIDIA API catalog
. To
sign in
, you’ll be prompted to enter a personal or business email address to access different options for building with NIM. For more information, see the
NVIDIA NIM FAQ
.
This post was originally published on 10/23/2024.
借助 NVIDIA NIM 智能体蓝图创建客户服务 AI 虚拟助理的三大核心组件
在当今快节奏的商业环境中，提供卓越的客户服务已不再是一种好事情，而是一种必要条件。无论是解决技术问题、解决计费问题，还是提供服务更新，客户都期望在方便时得到快速、准确和个性化的响应。然而，实现这种服务水平面临重大挑战。
传统方法，例如静态脚本或手动流程，通常在提供个性化和实时支持方面存在不足。此外，许多客户服务运营依赖于敏感和零碎的数据，这些数据受严格的数据治理和隐私法规的约束。随着生成式 AI 的兴起，公司旨在通过提高运营效率、降低成本和最大限度地提高投资回报率（ROI）来革新客户服务。
将 AI 集成到现有系统会带来透明度、准确性和安全性方面的挑战，这些挑战可能会阻碍采用并中断工作流程。为此，公司正在利用由生成式 AI 提供支持的虚拟助理来管理各种任务，进而缩短响应时间并释放资源。
本文概述了开发者如何使用
AI 虚拟助理 NVIDIA NIM 智能体蓝图
（NVIDIA NIM Agent Blueprint）通过生成式 AI 扩展客户服务运营。通过利用这些信息（包括示例代码），企业可以满足日益增长的对卓越客户服务的需求，同时确保数据完整性和治理。无论是改进现有系统还是创建新系统，此蓝图都能使团队通过高效、有意义的交互满足客户需求。
更智能的 AI 虚拟助理，以及使用检索增强型生成技术的 AI 查询引擎。
构建 AI 虚拟助理时，务必要与组织的独特用例要求、机构知识和需求保持一致。然而，传统机器人通常依赖于刚性框架和过时的方法，难以满足当今客户服务环境不断变化的需求。
在各行各业，基于 AI 的助理都可以发挥变革性作用。例如，电信公司以及大多数零售和服务提供商可以使用 AI 虚拟助理来增强客户体验，方法是每周 7 天、每天 24 小时提供支持，同时使用多种语言处理各种客户查询，并提供动态的个性化互动，以简化故障排除和账户管理。这有助于减少等待时间，并确保针对不同客户需求提供一致的服务。
另一个例子是在医疗保险付款人行业，确保积极的会员体验至关重要。虚拟助理通过为会员提供个性化支持来增强这种体验，解决他们的索赔、保险查询、福利和付款问题，同时确保遵守医疗健康法规。这还有助于减轻医疗健康工作者的行政负担。
借助 NVIDIA AI 平台，组织可以创建 AI 查询引擎，该引擎使用
检索增强生成（RAG）
将 AI 应用与企业数据相连接。借助 AI 虚拟助理 NIM Agent Blueprint，开发者可以快速着手构建可提供增强客户体验的解决方案。它使用以下
NVIDIA NIM
微服务构建：
NVIDIA NIM for LLM:
将先进的大语言模型 (LLM) 的强大功能引入应用，以非凡的效率提供出色的自然语言处理能力。
Llama 3.1 70B Instruct NIM
：凭借出色的上下文理解、推理和文本生成能力，为复杂对话提供支持。
NVIDIA NeMo
Retriever NIM:
通过此集合，您可以轻松访问作为 RAG 工作流基础模组的先进模型。这些工作流集成到虚拟助理解决方案时，可以无缝访问企业数据，通过快速、准确和可扩展的答案释放机构知识
NeMo 检索器嵌入（NIM）
：提升文本问答检索性能，为下游虚拟助理提供高质量的嵌入。
NeMo 检索器重新排名 NIM
：通过微调的重新排名器进一步增强检索性能，在查询 LLM 时找到要作为上下文提供的最相关的段落。
该蓝图旨在与现有的客户服务应用无缝集成，且不会破坏信息安全要求。得益于 NVIDIA NIM 的可移植性，组织可以在数据驻留的任何位置集成数据。通过将生成式 AI 引入数据，该架构使 AI 虚拟助理能够利用每位客户的独特配置文件、用户交互历史记录和其他相关数据，为其提供更个性化的体验。
蓝图是一个起点，可以根据企业的独特用例进行定制。例如，集成其他 NIM 微服务（例如
Nemotron 4 Hindi 4B Instruct
），使 AI 虚拟助理能够使用当地语言进行通信。其他微服务可以启用其他功能，例如合成数据生成和模型微调，以更好地满足您的特定用例需求。在将 AI 虚拟助理连接到数字人 NIM Agent 蓝图时，为其提供类似人类的界面。
通过实施包含专有数据（公司和用户资料及其特定数据）的 RAG 后端，AI 虚拟助理可以进行高度上下文相关的对话，实时满足每位客户的具体需求。此外，该解决方案可在现有的治理框架内安全运行，确保遵守隐私和安全协议，尤其是在处理敏感数据时。
用于创建自己的 AI 虚拟助手的三个构建块
作为开发者，您可以构建自己的 AI 虚拟助手，实时检索相关性极高的最新信息，并获得不断改进的人性化响应。图 1 显示了包含三个功能组件的 AI 虚拟助手架构图。
图 1. 客户服务 NIM Agent Blueprint 的 AI 虚拟助手
1.数据提取和检索管道
工作流管理员使用提取工作流将结构化和非结构化数据加载到数据库中。结构化数据的示例包括客户档案、订单历史记录和订单状态。非结构化数据包括产品手册、产品目录和支持材料（例如 FAQ 文档）。
2.AI 智能体
AI 虚拟助理是第二个功能组件。用户通过用户界面与虚拟助理交互。在 LangGraph 智能体 LLM 编程框架中实施的 AI 智能体计划如何处理复杂的客户查询并递归地解决问题。LangGraph 智能体使用
Llama 3.1 70B Instruct NIM
的工具调用功能从非结构化和结构化数据源检索信息，然后生成准确的响应。
AI 智能体还使用短期和长期记忆功能来实现多回合对话历史记录。内置主动对话查询和响应，以便稍后在对话中作为附加上下文进行检索。这使得客户与智能体的交互更像人类，并消除了客户重复已与智能体共享的信息的需求。
最后，在对话结束时，AI 坐席会总结讨论内容以及情感判断，并将对话历史记录存储在结构化数据库中。来自同一用户的后续交互可以在未来对话中作为附加上下文进行检索。呼叫摘要和对话历史记录检索可以缩短通话时间并改善客户体验。情感判断可以为客户服务管理员提供有关坐席有效性的宝贵见解。
3.运营管线
客户运营流程是整个解决方案的第三个功能组件。该流程为客户服务操作员提供重要信息和见解。管理员可以使用运营流程来查看聊天记录、用户反馈、情感分析数据和通话摘要。分析微服务利用 Llama 3.1 70B Instruct NIM，可用于生成平均通话时间、解决问题所需时间和客户满意度等分析。分析还可用作用户反馈，重新训练 LLM 模型以提高准确性。
与 NVIDIA 合作伙伴一起投入生产
NVIDIA 咨询合作伙伴正在帮助企业采用使用 NVIDIA 加速计算和
NVIDIA AI Enterprise 软件
构建的世界级 AI 虚拟助手，其中包括 NeMo、NIM 微服务和 NIM Agent 蓝图。
Accenture
Accenture AI Refinery
基于
NVIDIA AI Foundry
构建，可帮助设计自主、意图驱动的客户交互，使企业能够通过数字人或交互代理等创新渠道为个人定制旅程。具体的用例可以根据各个行业的需求进行定制，例如电信呼叫中心、保险政策顾问、制药互动代理或汽车经销商网络代理等。
Deloitte
德勤前线 AI 借助由 NVIDIA ACE、NVIDIA Omniverse、NVIDIA Riva 和 NIM 等 NVIDIA 技术加速的 NVIDIA NIM Agent 蓝图构建的数字虚拟形象和 LLM 代理，提升客户服务体验。
Wipro
Wipro Enterprise Generative AI (WeGA) Studio 加速特定行业的用例，包括医疗健康、金融服务、零售等领域的联系中心代理。
Tech Mahindra
Tech Mahindra 正在利用 NVIDIA 数字人 NIM Agent Blueprint 构建客户服务解决方案。借助 RAG 和 NVIDIA NeMo，该解决方案使学员能够在对话期间举起一只手提出澄清问题，从而阻止代理。该系统旨在通过改进的学习管理系统与后端微服务连接，该系统可部署在许多行业用例中。
Infosys
Infosys Cortex
是
Infosys Topaz
的一部分，是一个 AI 驱动型客户互动平台，集成了 NVIDIA NIM Agent 蓝图以及 NVIDIA NeMo、Riva 和 ACE 技术，可实现生成式 AI、语音 AI 和数字人功能，为客户服务组织的每位成员提供专业、个性化、前瞻性和按需协助，从而在增强客户体验、提高运营效率和降低成本方面发挥关键作用。
Tata 咨询服务
Tata 咨询服务 (TCS) 虚拟代理由 NVIDIA NIM 提供支持，并与 ServiceNow 的 IT 虚拟代理集成，旨在优化 IT 和人力资源支持。该解决方案使用 prompt-tuning 和 RAG 来改善响应时间、提高准确性，并提供多回合对话功能。优势包括降低服务台成本、减少支持工单、提高知识利用率、加快部署速度，以及改善员工和客户的整体体验。
Quantiphi
Quantiphi
正在将
NVIDIA NIM
Agent 蓝图集成到其对话式 AI 解决方案中，通过逼真的数字虚拟形象增强客户服务。这些先进的虚拟形象由 NVIDIA Tokkio 和 NVIDIA ACE 技术、NVIDIA NIM 微服务和
NVIDIA NeMo
提供支持，可与现有的企业应用无缝集成，从而以更高的真实感增强运营和客户体验。事实证明，针对数字虚拟形象工作流程进行微调的 NVIDIA NIM 部署极具成本效益，可减少企业在令牌方面的支出。
SoftServe
SoftServe Digital Concierge
由 NVIDIA NIM Agent 蓝图和 NVIDIA NIM 微服务加速，使用 NVIDIA ACE、NVIDIA Riva 和 NVIDIA Audio2Face NIM 微服务提供高度逼真的虚拟助手。借助 Character Creator 工具，它可以提供语音和面部表情，准确性和细节都非常逼真。
借助 NVIDIA NeMo Retriever 的 RAG 功能，SoftServe Digital Concierge 可以智能地回复客户查询，方法是参考上下文并提供特定、最新的信息。它可以将复杂的查询简化为清晰、简洁的答案，还可以在需要时提供详细的解释。
EXL
EXL 的智能体助手产品是一种利用 NVIDIA Riva、NVIDIA NeMo 和 NVIDIA NIM 微服务的联络中心 AI 解决方案。EXL 计划使用 AI 虚拟助手 NVIDIA NIM Agent 蓝图来增强其解决方案。
在本周的
NVIDIA AI 峰会印度大会
上，NVIDIA 咨询合作伙伴宣布与 NVIDIA 合作，将印度转变为 AI 前沿办公室。使用 NVIDIA 技术，这些咨询巨头可以帮助客户定制客户服务代理蓝图，使用他们喜欢的 AI 模型（包括来自印度模型制造商的主权 LLM）构建独特的虚拟助理，并在他们选择的基础设施上高效运行。
开始使用
要免费试用蓝图并查看系统要求，请导航至
Blueprint Card
。您还可以
注册，以便在
AI 虚拟助手 NIM Agent Blueprint 准备下载时收到通知。
要开始使用这些微服务构建应用程序，请访问
NVIDIA API Catalog
。要
登录
，系统会提示您输入个人或企业电子邮件地址，以访问使用 NIM 构建的不同选项。有关更多信息，请参阅
NVIDIA NIM 常见问题解答
。