LLAMA 3.1 70B, LLAMA 3.3 70B
Edgeless Systems
6 months
faster time to launch vs. on-premises AI projects50%
savings compared to on-premises AI deployments1,000s
of employees onboarded, and it’s just the beginning*All results are self-reported and not identifiably repeatable. Generally, expected individual results will differ.
CASE STUDY
LLM service uses confidential computing to deliver secure cloud AI applications
At a glance
Industry
TechnologyUse case
Developing secure, compliant AI applicationsGoal
Enable businesses across sectors to use cloud-based generative AILlama versions
Llama 3.1 70B Instruct AWQ INT4, Llama 3.3 70B Instruct AWQ INT4Deployment
Bare metal hosted cloud via ScalewayTHEIR STORY
Making cloud AI applications secure
Edgeless Systems is a German-based cybersecurity company that makes the public cloud the safest place for sensitive data. The company builds world-leading, open-source solutions for confidential computing, taking data security for cloud and AI applications to an unprecedented level, enabling encrypted data processing at scale.THEIR GOAL
Empower businesses to use generative AI and protect sensitive data
Businesses in heavily regulated industries like healthcare and banking were missing out on AI workplace solutions because cloud AI applications didn’t meet their strict data security requirements. Edgeless Systems set out to create a cloud-based service that could fortify chatbots and other AI tools using confidential computing — a technology that isolates sensitive data and computations to keep data encrypted at all times.
THEIR SOLUTION
A state-of-the-art LLM service that keeps data safe and secure
Edgeless Systems created Privatemode AI: the first end-to-end, cloud-based large language model (LLM) service that uses confidential computing to meet strict encryption requirements and mitigate security risks.Llama models rose to the top for PrivatemodeAI because they were open, multilingual and self-hostable, making them easily downloadable to run on Edgeless Systems’ platform. The company ultimately chose a quantized version — Llama 3.1 70B Instruct AWQ INT4 (later upgrading to Llama 3.3 70B Instruct AWQ INT 4) — to balance performance with the compute efficiency to run on a single NVIDIA H100 GPU, which was necessary to enable confidential computing.
Privatemode AI provides end-to-end encrypted AI services for heavily regulated industries.
THEIR APPROACH
Using confidential computing for AI applications
Privatemode AI creates a secure environment that separates the infrastructure and service providers from the data and models. A lightweight client proxy uses an API to plug directly into existing codebases, encrypts prompts and decrypts responses locally to ensure confidentiality.On the server side, secure components run inside a confidential container. The contrast coordinator manages attestation and credentials, while the secret service controls encryption key access. Verified AI workers run the Llama model securely, never exposing data in plaintext.
Privatemode AI verifies the integrity of each step of the confidential computing service.
THEIR SUCCESS
Companies with sensitive data can safely use generative AI in the cloud
Privatemode AI made it possible for industries such as finance, law and healthcare to safely take advantage of flexible cloud AI applications that enhance productivity. Businesses can quickly launch secure AI assistants, chatbots and more without investing in on-premises deployments — saving both time and money.
- 6 months faster time to launch vs. on-premises AI projects
- 50% savings compared to on-premises AI deployments
- 1,000s of employees onboarded, and it’s just the beginning
“Before Privatemode AI, tight security required extremely costly on-prem deployments. Now, companies have a secure option that uses confidential computing to isolate prompts and data from the underlying infrastructure and AI models. AI workloads can access powerful Llama models in the cloud with the same level of security and privacy.”
Moritz Eckert, VP of Product and Technology, Edgeless Systems
“Organizations across sectors like healthcare, finance and government can now use Privatemode AI and powerful Llama models in the cloud without compromising on security or privacy.”
Martin Paloncy, Partnerships Manager, Edgeless Systems
Previous slide
Next slide
Start building
Explore more
ConsumerBusiness
Shopify | Llama case studiesShopify uses Llama to generate product pages, localize content, and automate support, helping developers scale workflows and save time.Read more
ConsumerTech
Scribd, INC | Llama case studiesDelivering faster, cheaper and more accurate results with a Llama-powered AI content discovery assistant.Read more
Stay up-to-date
Our latest updates delivered to your inbox
Subscribe to our newsletter to keep up with the latest AI updates, releases and more.