The software can help developers guide generative AI applications to create impressive text responses that stay on track.
It was recently revealed that Nvidia released new software, LLAMADA NeMo Guardrails, to help developers and businesses guide and control the generative responses of their AIs.
Nemo Guardrails aims to make AI chatbots and other applications created from extensive language models (LLM) are accurate, appropriate, relevant and secure. In other words, will help users prevent AI models from indicating incorrect facts, go off topic, talk about dangerous topics or open security holes. The announcement suggests that NeMo Guardrails can help solve the AI chatbot's hallucinating problem, but many are skeptical about it.
Despite the hype, large text-generating AI models like OpenAI's GPT-4 do make a lot of mistakes, some of which are detrimental. And it is that as we know the AI are trained from terabytes of data to create programs capable of creating blocks of text that are read as if they were written by a human.
But they also tend to make things up, which practitioners often call "hallucination." Early applications of the technology, such as summarizing documents or answering basic questions, must minimize "hallucinations" to be useful.
The creators of these AI models claim to take action to address issues, such as setting up filters and teams of human moderators to fix issues when they are reported. But there is no single solution.
Even today's best models are susceptible to bias, toxicity, and malicious attack. In its quest for "more secure" LLMs, Nvidia released NeMo Guardrails, an open source toolset aimed at making AI-powered applications more accurate, appropriate, relevant, and secure. Nvidia said that its tool is the result of years of research.
According to Nvidia, NeMo Guardrails is a software layer that stands between the user and the LLM or other AI tools since removes bad results before the model produces them and prevents the model from processing incorrect indications.
In a statement, Jonathan Cohen, Nvidia's vice president of applied research, explained:
“If you have a customer service chatbot, designed to talk about your products, you probably don't want it to answer questions about our competitors. You want to monitor the conversation. And if that happens, you steer the conversation to the topics you prefer."
NeMo Guardrails includes code, examples, and documentation to add security to AI applications that generate text and speech. Nvidia says that the toolkit is designed to work with most LLMs, allowing developers to create rules using just a few lines of code.
“Ultimately, developers control what is outside the limits of their application with NeMo Guardrails. They can develop railings that are too wide or conversely too narrow for their use case,” Cohen said.
Nemo Guardrails allows developers to define three types of limits:
- Security measures Themes prevent apps from being rerouted to unwanted domains. For example, they prevent customer service assistants from answering questions about the weather.
- Security measures, they ensure that applications respond with accurate and appropriate information. They can filter out unwanted language and make sure references are only made to credible sources;
- Applications are limited to establish connections with external third-party applications that are known to be safe.
According to Nvidia, NeMo Guardrails can use an LLM to detect hallucinations by asking another LLM to check the first LLM's response.
It then returns "I don't know" if the model doesn't find any matching answers. Although the method seems practical, several reviewers have expressed skepticism towards it. In this regard, Nvidia has recognized that NeMo Guardrails is not a one-size-fits-all solution to language model deficiencies. The Santa Clara, California-based giant said its new set of tools is not without its flaws. In other words, NeMo Guardrails won't cover everything.
Cohen toon points out that NeMo Guardrails works much better with models that are good enough to follow instructions, such as ChatGPT, and which use the popular LangChain framework to build AI-powered applications.
This rules out some of the existing open source options. In addition to questions about the effectiveness of the tool, it should be noted that Nvidia doesn't necessarily release NeMo Guardrails out of sheer goodness. It's part of the company's NeMo framework, which is available through Nvidia's enterprise AI software suite and its fully managed NeMo cloud service.
Finally, if you are interested in being able to know more about it, you can consult the details in the following link