Reducing hallucinations with a better context layer for AI agents
At a glance
Industry
TechnologyUse case
Creating a no-code, agent-building solution grounded in enterprise dataGoal
Design better context engineering to reduce hallucinationsLlama versions
Llama 3.1 8B, Llama 3.3 70BDeployment
Google Cloud
Turning complex enterprise data into reliable AI solutions with context engineering
Founded by some of the industry’s earliest contributors to retrieval-augmented generation (RAG) techniques, Contextual AI specializes in context engineering for enterprise systems. The company’s modular solution is designed to allow anyone to quickly build RAG-based agents that retrieve information from enterprise data sources, with additional functionality for developers and power users.Enable enterprises to reduce hallucinations by building a more grounded context layer
Even when enabled with RAG capabilities, generative AI models still hallucinate sometimes. Contextual AI wanted to make it easier for enterprises to connect data sources to their solution and get high accuracy right out of the box. Minimizing hallucinations meant improving the context layer with a grounded language model that could check against source data to ensure responses were accurate. A grounded model responds only with relevant information from a dataset and gives inline attributions to the underlying data sources.
Fine-tuning Llama to beat foundational models
Initially, the Contextual AI team believed they would have to train their own grounded language model from scratch, which would have been time-consuming and capital-intensive. But after fine-tuning Llama 3.3, the team was able to beat every commercial foundational model in grounding performance. For the team’s purposes, Llama models have been easier than alternative models to fine-tune and achieve excellent accuracy, groundedness, latency and cost performance.In addition to the Contextual AI Grounded Language Model (GLM), the team also used Llama to develop reward models for their RAG solution, including LMUnit and a groundedness scoring model. LMUnit helps developers evaluate natural language models through fine-grained unit tests, which can streamline debugging. The groundedness scoring model reviews all generated answers to rate adherence to the prompt and context, acting as a final layer of protection against hallucination.
A unified system for RAG components
To build an enterprise’s context layer, Contextual AI starts by ingesting not just structured data but also thousands, or even millions, of multimodal documents. These are often highly technical and can include diagrams, schematics, charts and tables. Once this information is in the datastore, retrievers and rerankers optimize which sources are prioritized and fed into the generative model. The GLM can then respond to a wide range of queries.The entire pipeline runs on Google Cloud, which includes training on GPUs, inference on Vertex AI, and multimodal data ingest using Gemini 2.5 Flash and Pro.With APIs that orchestrate RAG components as a unified system, Contextual AI helps enterprises scale from early projects to more sophisticated use cases. Developers who want more control can also access primitives for document parsing, reranking, grounded generation and LLM evaluation. This unified approach gives enterprises precise control over which data powers their AI agents.
Faster time to value-building AI agents
Today, Contextual AI’s solution can derive answers from enormous volumes of complex, highly technical data that would otherwise be impossible for a single human to review. Users can build custom agents capable of supporting PhD-level engineers and providing answers within minutes. That makes the solution especially popular with enterprises in finance, legal services and technology and engineering. More than a dozen partners are already building vertical solutions on the platform.Having achieved the highest score for FACTS Grounding, the GLM from Contextual AI is the most hallucination-resistant and factual model currently available. It’s part of an enterprise-grade solution that offers scalability, security and governance.
