Run inference

Run Muse Glimmer on your own infrastructure or call it through a hosted cloud provider. The self-hosted runtimes below load the released checkpoints (or a quantized version of them) and use the Muse Glimmer chat format described in the prompting guide.

RuntimeBest forHardwareServing API
vLLMProduction serving, high throughputNVIDIA GPU (single or multi)OpenAI-compatible HTTP
SGLangHigh-throughput serving for concurrent usersNVIDIA GPU, Apple siliconOpenAI-compatible HTTP
llama.cppLocal and mixed CPU/GPU, laptopsCPU, NVIDIA/AMD GPU, Apple MetalOpenAI-compatible HTTP + CLI
ExecuTorchAhead-of-time export, local servingNVIDIA GPU on Linux or Windows, Apple silicon on macOSOpenAI-compatible HTTP
Stream reasoning workloads

Muse Glimmer produces long chain-of-thought reasoning by default. For reasoning workloads, request streaming (stream: true) so long generations don't hit request timeouts. See prompting and Deploy with vLLM.

Deploy with vLLMProduction-grade throughput with an OpenAI-compatible endpoint.
Deploy with SGLangHigh-throughput serving for concurrent users.
Deploy with llama.cppRun locally on CPU, GPU, or a mix.
Deploy with ExecuTorchExport ahead of time for CUDA or Apple silicon.

Run inference on cloud

Use Together AI to call Muse Glimmer through a hosted API without managing model infrastructure.

Run with Together AISet up authentication and send your first hosted inference request.

Next steps

Once you have picked a runtime or provider, tune your prompts with the prompting guide, shrink the model with quantization, or speed up self-hosted generation with speculative decoding.