Nvidia, a leader in AI and accelerated computing, announced the general availability of its NeMo microservices on Wednesday, April 23, 2025, marking a significant step toward enabling enterprises to develop custom AI agents. These software tools, part of Nvidia’s AI Enterprise software portfolio, are designed to create “agentic” AI systems that act as digital employees, enhancing productivity across industries such as call centers, software development, and customer service.
A New Era of Agentic AI
Joey Conway, Nvidia’s senior director of generative AI for enterprise, described NeMo microservices as a way to integrate AI agents into enterprise workflows, likening them to digital teammates. “There are over a billion knowledge workers across many industries, geographies, and locations,” Conway said during a media briefing. “Our view is that digital employees, or AI agents, will help enterprises get more work done in these various domains and scenarios.”
The NeMo microservices include key components like NeMo Customizer, Evaluator, Guardrails, Retriever, and Curator. These tools simplify the process of building, optimizing, and scaling AI agents. For instance, NeMo Customizer accelerates large language model (LLM) fine-tuning with up to 1.8x higher training throughput, while NeMo Evaluator streamlines model assessment with just five API calls. NeMo Guardrails enhances compliance protection by 1.4x with minimal latency, ensuring safety and security in AI operations.
Must Read: Microsoft Rolls Out Controversial Recall Feature for Windows 11 Copilot+ PCs
Data Flywheels for Continuous Improvement
Central to NeMo’s capabilities is the concept of a “data flywheel,” a continuous learning cycle where AI agents improve through human and AI feedback, as well as enterprise data. This approach ensures that agents remain accurate and up-to-date, transforming raw data into actionable insights. Nvidia’s NeMo Retriever, for example, enables developers to build complex data pipelines, such as retrieval-augmented generation (RAG), to extract and process information efficiently.
The microservices support a wide range of open AI models, including Meta’s Llama, Google’s Gemma, Microsoft’s Phi, Mistral, and Nvidia’s own Llama Nemotron Ultra, which excels in scientific reasoning, coding, and complex math benchmarks. This flexibility allows enterprises to tailor AI agents to specific use cases, from fraud detection to inventory management.
Real-World Impact and Partnerships
Several high-profile companies are already leveraging NeMo microservices to enhance their operations. AT&T, in collaboration with Arize and Quantiphi, used NeMo Customizer and Evaluator to fine-tune a Mistral 7B model, improving AI agent accuracy for personalized services and fraud prevention. BlackRock is integrating NeMo into its Aladdin platform to unify investment management, while Cisco’s Outshift unit and Galileo have built a coding assistant with 10x faster response times and 40% fewer tool selection errors.
Nvidia has also partnered with leading AI software providers like Cloudera, Datadog, Dataiku, DataRobot, and DataStax, integrating NeMo microservices into their platforms. Developers can access these tools through popular AI frameworks such as CrewAI, LangChain, LlamaIndex, and Llamastack, making it easier to deploy AI agents across cloud, data center, and on-premises environments.
Must Read: Top Gaming Laptops Under $2,500 for 2025: A Comprehensive Comparison
Challenges and Opportunities
While NeMo microservices promise to revolutionize enterprise productivity, challenges remain. Recent research from the UK, commissioned by Storyblok, found that 44% of business leaders reported only slight improvements from AI investments, despite an average spend of £321,000 ($427,000). However, 93% of respondents claimed their AI investments delivered good ROI, suggesting that deeper integration, as facilitated by tools like NeMo, could unlock greater value.
Nvidia’s NeMo microservices are now available through the Nvidia NGC catalog or as part of the Nvidia AI Enterprise suite, offering enterprise-grade security, stability, and support. As businesses increasingly adopt agentic AI, Nvidia’s tools position them to build scalable, efficient, and secure AI systems that drive meaningful transformation.