Helping clients extract objective analyses from complex documents and records
Case-Based Research (CBR) is a pseudonym for a successful SaaS company that uses AI and machine learning (ML) to extract salient details from multiple document types and create objective research reports.
THEIR GOAL
Automate the most difficult research cases with Generative AI
Research is a challenging, time-consuming task for human experts. A thorough manual review of a single research case can take hours of an expert’s time.The CBR platform automates research services for millions of cases a month. A machine learning model handled 98% of research cases, but the remaining 2% were too complex to process, leaving clients with tens of thousands of cases to review by hand.Even expert professionals struggled to review these outlying cases, which could have more than 230 distinct data categories and noisy documentation. The team hoped large language models could do the job within CBR’s budget and SLAs.
THEIR SOLUTION
Fine-tuned Llama cracked the hardest research cases
CBR repowered its platform with a fine-tuned Llama 3 8B Instruct model. The model reached 90% accuracy on the most difficult research cases, outperforming the leading 4th generation LLM in accuracy and response times. By using a lightweight, small language model (SLM), CBR met their platform’s low-latency SLAs with far less computing overhead and far lower costs than a commercial model.
THEIR APPROACH
Predibase delivers fine-tuned models at scale
CBR used Predibase for training and production. The platform uses Low-Rank Adaptation (LoRA) and Parameter Efficient Fine-Tuning (PEFT) to deliver fine-tuned performance without retraining the entire model. This technique drastically reduces training time and computing overhead in production.After fine-tuning on a 150,000-row dataset, Llama 3 8B Instruct delivered 90% accuracy in production and maintained 0.15 sec response times under full-scale traffic loads.
THEIR SUCCESS
Llama delivers fast, objective analysis for complex research cases
With Llama 3 8B Instruct and the Predibase platform, CBR has automated its most difficult research cases, saving clients hours of expert labor and ensuring that complex research cases receive thorough, unbiased reviews.Projected results:
97%+ accuracy on the easiest 98% research cases
90%+ accuracy on the most difficult 2% of research cases
“Using Llama and Predibase, we've achieved lightning-fast inference and reduced costs by 5x compared to the leading 4th generation LLM. Most importantly, we’ve built a better product for our customers, leading to more objective, transparent and efficient research.”
Staff ML Engineer, CBR
“By using Predibase to fine-tune and serve a Llama 3 8B Instruct model, we achieved better accuracy, faster response times and 5x reduction in costs compared to traditional generative AI approaches utilizing commercial LLMs.”
Staff ML Engineer, CBR
“Our best-performing model was Llama 3 8B Instruct, which we fine-tuned on Predibase. We achieved an accuracy of 90% for the most challenging 2% of cases, outperforming both the leading 4th generation LLM and all of our other fine-tuning experiments. Additionally, Predibase consistently provided highly accurate results, whereas some of the other tools we tested were less reliable across fine-tuning runs.”
Shopify | Llama case studiesShopify uses Llama to generate product pages, localize content, and automate support, helping developers scale workflows and save time.Read more
ConsumerTech
Scribd, INC | Llama case studiesDelivering faster, cheaper and more accurate results with a Llama-powered AI content discovery assistant.Read more
Tech
Exati | Llama case studiesTransforming support for smart city platform clientsRead more
During evaluation, fine-tuned Llama 2 7B delivered 9.4% higher accuracy and 30x faster response times than the leading 4th generation LLM
CBR uses increasingly sophisticated classification services to process information efficiently.
5x faster onboarding for semiconductor professionals
30x faster response times in production versus the leading 4th generation LLM tests
*All results are self-reported and not identifiably repeatable. Generally expected individual results will differ.
Stay up-to-date
Our latest updates delivered to your inbox
Subscribe to our newsletter to keep up with the latest AI updates, releases and more.
