Chip giant Nvidia on Wednesday announced the general availability of tools to develop “agentic” artificial intelligence for enterprises.
Called NeMo microservices, the software tools, which are part of Nvidia’s AI Enterprise software portfolio, offer several functions that customize and repeatedly optimize the functioning of AI agents for a variety of tasks, including call centers and software development.
Also: The 4 types of people interested in AI agents – and what businesses can learn from them
In a media briefing, Nvidia’s head of generative AI for enterprise, Joey Conway, framed the NeMo software as a way to use AI agents as “digital employees.”
“Our view of where we see things going is that there are over a billion knowledge workers across many industries, geographies, and locations,” said Conway. “And our view is that digital employees, or AI agents, will be able to help enterprises get more work done in these various domains and scenarios.”
Productivity gains
The early implementations of the AI agents have demonstrated measurable productivity gains, said Conway.
For example, Amdocs, a maker of software used by phone companies, has used NeMo microservices to create billing agents, sales agents, and network agents. The billing agent, which handles customers’ calls about their phone bills, was able to resolve more inquiries, including a 50% increase in what’s called “first-call resolution,” said Conway.
Also: Crawl, then walk, before you run with AI agents, experts recommend
Conway’s remarks about agents as digital employees echo a persistent theme from the past year: the idea of AI code as corporate “workers” that can take over corporate processes and be managed just like employees.
Nvidia has been offering NeMo software for over five years in a variety of forms, with the overarching goal of speeding up companies’ development of AI models.
Also: 5 ways to boost your team’s productivity – without relying on generative AI
Along the way, the company in 2022 began offering NeMo pre-built AI models as an on-demand cloud offering. The microservices followed in October of last year.
<!–>
New microservice components
Components of NeMo include two microservices that have already been available: Curator and Retriever. Curator is used by developers to build “pipelines” that clean and refine data sets used to train or fine-tune AI models. Retriever takes data sources and extracts elements that will be used by the model, such as text, graphics, and chart elements.
Also: 93% of IT leaders will implement AI agents in the next two years
Three additional components work with Curator and Retriever: Customizer, Evaluator, and Guardrails.
The Customizer microservice takes output from Curator and combines it with techniques for post-training, or fine-tuning, to “teach these models new skills,” as Conway put it.
The Evaluator is a sort of push-button version of AI benchmark tests, which run the model through testing after it has been through Customizer, to evaluate whether the model “actually improved and gained new skills.”
Also: As AI agents multiply, IT becomes the new HR department
Guardrails is meant to operate at runtime with the AI agent to improve “compliance protection” with respect to “safety and security measures” for an enterprise.
Updating and gaining new abilities
The intention with NeMo is that models pass repeatedly through the various microservices to be updated and gain new abilities, what Nvidia refers to as a “flywheel.”
The NeMo microservices are paired with Nvidia’s infrastructure software for deployment of agents, called NIM, an acronym for Nvidia Inference Microservices. A NIM is an AI model in an application container that runs on a container manager, such as Kubernetes, and is accessed by developers via an API.
Also: I’ve tried lots of AI image generators, and Nvidia and MIT’s is the one to beat for speed
The NeMo software will greatly simplify many of the tasks of training, post-training, evaluating, and revising that developers have to do if they work directly with Python code and AI frameworks, said Nvidia’s Conway.
“The focus for NeMo microservices is being able to build these microservices so that the rest of the ecosystem can get started much faster,” said Conway. “From our experience, we’ve seen that these can be quite complicated,” he said, referring to developing AI models and agents, and deploying them.
Also: 5 ways to manage your team more effectively in the AI-enabled enterprise
“Previously, many of our advanced customers had to rely on various open-source libraries, which are often best effort and not always correct,” he added. “We’ve been able to take all of that software, put it under NeMo Evaluator, add the latest techniques, and then simplify the interaction so it’s a few simple API calls.”
Get the morning’s top stories in your inbox each day with our Tech Today newsletter.
