Shopify is an all-in-one commerce platform that powers millions of businesses. By making it easier to start, run and grow a business, Shopify helps people around the world achieve independence through commerce.
THEIR GOAL
Improving listing accuracy with image-aware AI
Shopify merchants upload millions of product images to the platform every day. Each product image contains valuable information including the product’s color, material, size and more.As multi-modal models matured, Shopify developers saw a new way to help merchants create high-performance listings: use the visual information inside product images to help merchants create more effective product attributes.
THEIR SOLUTION
Dual real-time and offline agents automate product tagging and categorization
Faced with more than 10,000 possible attribute categories, millions of new products each day and billions of existing images, Shopify developers realized they needed to split the solution into two use cases.A real-time AI agent helped merchants create accurate product attributes as they entered new product listings. A second offline agent extracted attributes from the billions of images and product descriptions on the platform and updated listings for better search performance.
Open-source LLaVA delivered the right mix of capabilities and low cost
THEIR APPROACH
Quantized training saved time and reduced computing costs
The Shopify team needed higher performance than stock LlaVA could deliver, so developers retrained a Llama 2 7B-based LLaVA model on a mix of real-world and synthetic data, all vetted by human evaluators.The team leveraged Quantized Low-rank Adaptation (QLoRA) for the first training rounds. QLoRA used quantization and selective parameter updating to reduce memory footprints and compute workloads so training could run on more modest hardware.Training Llama 7B at lower precision with QLoRA got predictions within 95% of the required accuracy. To reach their final target, the team switched to full-precision training on bare metal resources in the Google Cloud Platform.
THEIR SUCCESS
Happier merchants, more accurate product attributes
The Llama-powered merchant assistant made it far easier to correctly categorize product listings and create accurate attributes. The offline solution delivered platform-wide improvements as it reviewed billions of images, improved descriptions and added nuance by applying more than 10,000 possible metadata categories.The improved listings delivered major benefits to shoppers. Products surfaced accurately, so shoppers were able to find what they were looking for fast — and that increased sales sitewide.
“Closed models performed well, but their pricing structures made them prohibitively expensive. When you need to run billions of multi-modal inferences a day, per-token charges aren’t sustainable.”
“We were pleasantly surprised to find that with fine-tuning, the smaller Llama 7B model delivered the accuracy we needed. Using a smaller parameter size sped up training and saves compute cycles in production.”
Agatha Krajewski, Director of Product, Shopify
“Analyzing every image on the platform and generating metadata is an immense task. Using smaller-parameter Llama models makes it possible without expensive, bleeding-edge hardware.”
“Without Llama-powered, open-source LLaVA, providing real-time listing support for merchants would be prohibitively expensive, let alone optimizing Shopify's entire product catalog. Llama 2 7B's small footprint and zero per-token fees made our project both technically and economically feasible.”
Scribd, INC | Llama case studiesDelivering faster, cheaper and more accurate results with a Llama-powered AI content discovery assistant.Read more
Tech
Exati | Llama case studiesTransforming support for smart city platform clientsRead more
Tech
Oxide AI | Llama case studiesFinding trustworthy signals in a sea of financial dataRead more
Developers began with proprietary models, but per-token charges made them impractical for processing billions of tokens per day. Llama-based Large Language and Vision Assistant (LLaVA) — an open-source multi-modal model that combines a vision encoder with Llama 2 — delivered competitive results without per-token charges, making it an ideal choice for production inference at a massive scale.
Shopify retrained LLaVA to extract product attributes from images, create uniform metadata, categorize listings and provide conversational search services for customers.
Tens of billions of tokens processed daily
Improved merchant experience
More accurate product attributes and metadata
Intelligent customer search and discovery
*All results are self-reported and not identifiably repeatable. Generally expected individual results will differ.
Stay up-to-date
Our latest updates delivered to your inbox
Subscribe to our newsletter to keep up with the latest AI updates, releases and more.
