Customization

Adapt Muse Glimmer to your domain in two stages, mirroring how the released model was post-trained: supervised fine-tuning (SFT) teaches the model your task format and style from labeled examples, and reinforcement learning (RL) optimizes behavior against a reward signal — preference data or a programmatic reward — where labeled targets alone aren't enough.

Most projects start (and often stop) at SFT. Use RL when you need to optimize an outcome you can score but can't fully demonstrate: preference alignment, tool-use success, or reward-shaped agentic behavior.

StageWhat it doesWhen to useData you need
Fine-tuning (SFT)Trains on labeled input→output examples (LoRA, QLoRA, or full)Domain adaptation, style, format, instruction followingPrompt/response pairs in chat format
Reinforcement learningOptimizes against a reward signal (preference or programmatic)Preference alignment, tool-use success, agentic behaviorA reward model, preference pairs, or a verifiable reward
SFT first, then RL

SFT and RL work together. The standard recipe layers them — supervised fine-tuning first for well-formatted outputs, then reinforcement learning on that checkpoint, where RL is far more stable.

Fine-tuning (SFT)Adapt Muse Glimmer to your domain with LoRA, QLoRA, or full fine-tuning.
Reinforcement learningOptimize behavior against preference data or a programmatic reward.

Next steps

Prepare your data and run your first supervised fine-tune. When you have an SFT checkpoint that behaves well, layer on reinforcement learning. Deploy the result with any runtime.