Creating cutting-edge custom models
Neospace used Meta training techniques and Llama models to build custom large language models (LLMs) from scratch — a prohibitively expensive undertaking before open-source AI.At a glance
Industry
TechnologyUse case
Training and fine-tuning custom modelsGoal
Reduce model training time and cost while improving accuracyLlama version
Llama 3.1 405B Instruct, 70B and 8BDeployment
Oracle Cloud Infrastructure (OCI)
Shaping the next era of banking with AI
Neospace is a startup developing custom LLMs and SaaS for enterprise transformation, specifically expert models for complex customer support interactions, near-real-time personal offers and next-best actions based on customer history, credit scores and behavior.Create custom LLMs for complex domains and tasks
Neospace needed to create domain-expert models that could outperform commercial services like GPT, Anthropic and Gemini. Reaching that level of sophistication required training LLMs from scratch, an astronomically expensive proposition for a small startup. To succeed, they needed to find an alternative to full-weight model training.
Llama 3 provided an open-source blueprint for lower-cost, self-evolving models
Neospace used Meta’s post-training techniques as the foundation for rebuilding their training processes and redesigning their platform. With the new training system, Neospace expert models automatically improve, adapt and drive significant innovation. Their models include NeoLang for complex queries, NeoCredit for credit analysis and NeoFin for near-real-time customer intelligence, personalization and smart user experiences.
Transfer learning reduced training time 16x
Neospace used transfer learning to initialize new custom models. By initializing its models with Llama weights, Neospace reduced pretraining workloads from 16 trillion tokens to one trillion, which equates to 16x less computing time and expense.The team fine-tuned Llama 3.1 405B Instruct on financial mathematics, investment advisory and other industry-specific topics, then used it to create domain-specific, synthetic training data. After training, Llama 3.1 70B- and 8B-based Neospace models demonstrated precise, accurate behavior in complex financial use cases.
Open-source Llama helps Neospace take on giants
By building with Llama, Neospace rapidly brought its NeoLang, NeoCredit and NeoFin models to market at a fraction of the cost it would take to train a single model from scratch. The cost and time savings allow Neospace to create new models for new domains and continuously improve their solutions’ accuracy and performance.“We view Meta’s decision to release Llama as open source as a groundbreaking milestone in the history of AI,” says Neospace Founder and Chief Operations Officer, Gustavo Debs. “Making Llama openly accessible empowers organizations like ours to compete with industry leaders such as GPT and Anthropic.”
