Meta Global AI Developer Hackathon. $1M in prizes.Apply Now
LLAMA 3.1 8B INSTRUCT
Enterprise Consulting Partners
25
million annual queries7%
more accurate than GPT‑4o mini after fine-tuning4
second round trip time1
million hours saved*All results are self-reported and not identifiably repeatable. Generally expected individual results will differ.
CASE STUDY
Building a better AI assistant for nuanced knowledge work
At a glance
Industry
Professional servicesUse case
Improve existing AI assistant’s semantic understanding, accuracy and speedGoal
Transform information gathering and synthesisLlama version
Llama 3.1 8B Instruct with low-rank adaptation (LoRA)Deployment
PredibaseTHEIR STORY
Helping organizations understand complex situations and move ahead
Enterprise Consulting Partners (ECP) is a pseudonym for a leading professional services firm specializing in risk, strategy and people. The firm helps corporate and public sector leaders navigate an increasingly dynamic environment by addressing the most complex challenges of our time.
THEIR GOAL
Upgrade a confused AI agent with more nuanced language processing
ECP was an early adopter of generative AI, and its first AI assistant transformed how the firm’s analysts and knowledge workers navigated its extensive knowledge bases. However, as ECP integrated the AI assistant into enterprise platforms, it began to struggle with nuanced tasks.The ECP team explored creating expert models for specific tasks, but fully retraining multiple models introduced unmanageable complexity and was prohibitively expensive.
THEIR SOLUTION
Retool the AI assistant with Llama and LoRA
The ECP team used LoRA to create fine-tuned adaptations of Llama 3.1 8B Instruct. The adaptations load at run time on demand, which allows a single Llama model to generate responses for multiple fine-tuned adaptations at enterprise scale.With Llama and LoRA, ECP’s AI agent can bring expert AI assistance to unstructured institutional knowledge, current research and enterprise tools. Llama’s superior semantic understanding, multimodal skills and 128K token window are major upgrades to the AI assistant’s base capabilities, while LoRA adapters deliver task- and tool-specific expertise.
THEIR APPROACH
Fine-tune expert agents with less training effort
With fine-tuning, smaller models like Llama 3.1 8B Instruct can exceed the performance of general-purpose models with 10x or 100x the parameters. However, fully retraining smaller models is still a time-consuming and expensive process.LoRA uses adaptations to modify a fraction of a model’s weights, reducing fine-tuning time 5x to 10x. In production, ECP runs a single Llama 3.1 8B Instruct instance and uses LoRA Exchange (LoRAX) to launch adaptations on demand. LoRAX can spin up multiple fine-tuned LoRA adaptations simultaneously in the same basic computing footprint of Llama 3.1 8B Instruct. The solution can scale up to serve ECP’s global workforce without incurring massive computing bills.
LoRAX serves up fine-tuned adapters on demand, allowing a single model to deliver multi-agent services with a fraction of the computing costs.
THEIR SUCCESS
Expert AI agents save teams over one million hours
The combination of Llama and fine-tuned LoRA adaptations transformed ECP’s AI assistant. It has shouldered an astonishing amount of knowledge work, freeing ECP teams to focus on higher-order thinking and spend more time serving clients.Projected results:
- 25 million annual queries
- 7% more accurate than GPT‑4o mini after fine-tuning
- 4-second round trip time
- 1 million hours saved
“Llama’s impressive speed and ability to efficiently manage enterprise-scale queries have made our AI assistant more responsive, while LoRA fine-tuning is increasing its understanding, expertise and accuracy. Our upgraded AI assistant will deliver rapid access to vast stores of institutional knowledge and take on mundane tasks, improving overall operational efficiency.”
Global Chief Information Officer, Enterprise Consulting Partners
“With Llama and Predibase, the entry price and complexity dropped. Suddenly, we didn’t need separate infrastructure for every fine-tuned model. Training became incredibly cost-effective — tens of dollars, not hundreds of thousands. This unlocked a new wave of automation use cases that previously weren’t economical.”
Global Chief Information Officer, Enterprise Consulting Partners
“Our journey with generative AI started before it became a mainstream tool. With Llama, we upgraded to a fast, accurate, expert tool that upwards of 85,000 global employees tap into every day.”
Global Chief Information Officer, Enterprise Consulting Partners
Previous slide
Next slide
Start building
Explore more
ConsumerBusiness
Shopify | Llama case studiesShopify uses Llama to generate product pages, localize content, and automate support, helping developers scale workflows and save time.Read more
ConsumerTech
Scribd, INC | Llama case studiesDelivering faster, cheaper and more accurate results with a Llama-powered AI content discovery assistant.Read more
Stay up-to-date
Our latest updates delivered to your inbox
Subscribe to our newsletter to keep up with the latest AI updates, releases and more.