Run inference

Run Muse Spark Open on your own infrastructure or call it through a hosted inference provider. The self-hosted runtimes load the released checkpoints and use the Muse Spark Open chat format described in the prompting guide.

Runtime	Best for	Checkpoints	Serving API
SGLang	High-throughput serving for concurrent users	BF16, NVFP4	OpenAI-compatible HTTP
vLLM	Production serving, high throughput	BF16, NVFP4	OpenAI-compatible HTTP
TensorRT-LLM	Optimized serving on NVIDIA Blackwell, on a single 4-GPU node	NVFP4	OpenAI-compatible HTTP
NVIDIA Dynamo	Disaggregated serving and KV-aware routing	NVFP4	OpenAI-compatible HTTP
Hugging Face Transformers	Native Python implementation for research and custom pipelines	BF16, NVFP4	Python API
llama.cpp	Portable inference across Metal, CUDA, ROCm, Vulkan, and CPU	ggml-org GGUFs	OpenAI-compatible HTTP
Unsloth	Local inference on macOS, Windows, Linux, and WSL	Unsloth Dynamic GGUFs	OpenAI- and Anthropic-compatible HTTP
MLX	Inference on Apple silicon, across two or four Mac Studio computers	NVFP4	OpenAI-compatible HTTP

To call the model without provisioning GPUs, use a hosted provider.

Muse Spark Open is a mixture-of-experts model that serves across multiple GPUs. NVFP4 serves at TP4 on 4 × B300 and BF16 at TP8 on 8 × B300. Read Choose a checkpoint before you provision hardware.

Self-host

Each tab is a quick start for one runtime: a command that starts a server or, for Transformers, a script that loads the model and generates a reply. Each runtime's section below covers prerequisites, checkpoint variants, and sending requests.

bash
sglang serve \
  --model-path meta-models/Muse-Spark-Open-816B-A42B-NVFP4 \
  --tp-size 4 \
  --ep-size 4 \
  --quantization modelopt_fp4 \
  --moe-runner-backend flashinfer_trtllm \
  --mem-fraction-static 0.8 \
  --host 0.0.0.0 \
  --port 30000 \
  --enable-layernorm-sp \
  --enable-megamoe-for-layernorm-sp \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path RadixArk/Muse-Spark-Open-DFlash2 \
  --speculative-draft-attention-backend fa4 \
  --speculative-num-draft-tokens 8 \
  --mm-feature-transport cpu \
  --enable-hierarchical-cache \
  --hicache-size 200 \
  --reasoning-parser muse \
  --tool-call-parser muse

bash
export VLLM_USE_RUST_FRONTEND=1
vllm serve meta-models/Muse-Spark-Open-816B-A42B-NVFP4 \
    --tensor-parallel-size 4 \
    --enable-expert-parallel \
    --max-model-len 1048576 \
    --reasoning-parser muse \
    --tool-call-parser muse \
    --enable-auto-tool-choice \
    --speculative-config '{"method":"dspark","model": "Inferact/Muse-Spark-Open-DSpark","num_speculative_tokens":7,"draft_sample_method":"probabilistic","rejection_sample_method":"block","enable_adaptive_verification":true}'

bash
# serve_config.yaml: see TensorRT-LLM below
MODEL=meta-models/Muse-Spark-Open-816B-A42B-NVFP4   # or a local checkpoint path
trtllm-serve ${MODEL} \
    --host 0.0.0.0 --port 8000 \
    --config serve_config.yaml \
    --tool_parser musespark \
    --reasoning_parser musespark

python
# Launch with torchrun, which sets WORLD_SIZE; see Hugging Face Transformers below
import os
from transformers import AutoModelForMultimodalLM, AutoProcessor, DistributedConfig
distributed_config = DistributedConfig(
    tp_size=int(os.environ["WORLD_SIZE"]),
    ep_size=int(os.environ["WORLD_SIZE"]),
)
processor = AutoProcessor.from_pretrained("meta-models/Muse-Spark-Open-816B-A42B")
model = AutoModelForMultimodalLM.from_pretrained(
    "meta-models/Muse-Spark-Open-816B-A42B",
    distributed_config=distributed_config,
)
messages = [
    {"role": "user", "content": "Explain in one sentence why the sky is usually blue."},
]
inputs = processor.apply_chat_template(
    messages,
    reasoning_effort="low",  # default: xhigh, options: low/medium/high/xhigh
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device).to(model.dtype)
input_len = inputs["input_ids"].shape[-1]
outputs = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))
# to=self<|message|>We need to explain in one sentence why sky is usually blue.
# ...
# <|eom|><|start|>assistant to=user<|message|>The sky appears blue because air molecules scatter shorter-wavelength blue light from the sun more strongly than longer-wavelength red light.<|eot|>

bash
llama serve -hf ggml-org/Muse-Spark-Open-816B-A42B-GGUF

bash
unsloth run \
  --model unsloth/Muse-Spark-Open-816B-A42B-GGUF:UD-Q2_K_XL \
  -H 127.0.0.1 \
  -p 8888 \
  --context-length 32768 \
  --temperature 1.0 \
  --top-p 1.0 \
  --min-p 0.0 \
  --reasoning-effort medium

bash
# hostfile.json lists every Mac; see the MLX section below
mlx.launch --hostfile hostfile.json --backend jaccl --env MLX_METAL_FAST_SYNCH=1 -- \
  mlx_lm.server --model meta-models/Muse-Spark-Open-816B-A42B-NVFP4 --port 8080

The SGLang and vLLM tabs both serve NVFP4 on 4 × B300; the SGLang tab adds MegaMoE. Each runtime's section below has the other variants.

SGLang

Serve Muse Spark Open with SGLang(opens in new tab), an inference framework for low-latency, high-throughput serving on anything from a single GPU to a large distributed cluster.

Links

SGLang blog post on Muse Spark Open(opens in new tab): the features SGLang supports for Muse Spark Open at launch.

SGLang cookbook page for Muse Spark Open(opens in new tab): detailed deployment options.

DFlash2 draft model(opens in new tab): a DFlash2 checkpoint trained for speculative decoding with Muse Spark Open.

Install

Run Muse Spark Open with this Docker image:

bash
docker pull lmsysorg/sglang:dev-muse-spark-open

For host setup, see the SGLang installation documentation(opens in new tab).

Start the server

To serve NVFP4 checkpoint:

bash
sglang serve \
  --model-path meta-models/Muse-Spark-Open-816B-A42B-NVFP4 \
  --tp-size 4 \
  --ep-size 4 \
  --quantization modelopt_fp4 \
  --moe-runner-backend flashinfer_trtllm \
  --mem-fraction-static 0.8 \
  --host 0.0.0.0 \
  --port 30000 \
  --enable-layernorm-sp \
  --enable-megamoe-for-layernorm-sp \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path RadixArk/Muse-Spark-Open-DFlash2 \
  --speculative-draft-attention-backend fa4 \
  --speculative-num-draft-tokens 8 \
  --mm-feature-transport cpu \
  --enable-hierarchical-cache \
  --hicache-size 200 \
  --reasoning-parser muse \
  --tool-call-parser muse

To serve BF16 checkpoint:

bash
sglang serve \
  --model-path meta-models/Muse-Spark-Open-816B-A42B \
  --tp-size 8 \
  --ep-size 8 \
  --dtype bfloat16 \
  --mem-fraction-static 0.85 \
  --host 0.0.0.0 \
  --port 30000 \
  --enable-layernorm-sp \
  --enable-megamoe-for-layernorm-sp \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path RadixArk/Muse-Spark-Open-DFlash2 \
  --speculative-draft-attention-backend fa4 \
  --speculative-num-draft-tokens 8 \
  --mm-feature-transport cpu \
  --enable-hierarchical-cache \
  --hicache-size 200 \
  --reasoning-parser muse \
  --tool-call-parser muse

Recommended settings

For the best prefill performance on the NVFP4 checkpoint, run prefill with sequence parallelism and the MegaMoE kernel (--enable-layernorm-sp --enable-megamoe-for-layernorm-sp, which both server commands above already pass). Prefill is sharded across the four ranks and its MoE layers run on MegaMoE; decode stays on the TensorRT-LLM MoE kernel.

To tune it, add --attention-backend trtllm_mha --page-size 128 to the NVFP4 server command above, and set the MegaMoE token limit before starting it:

bash
export SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8192

SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK sets the maximum number of tokens that triggers MegaMoE. Raising it can improve prefill performance at larger batch sizes, but uses more device memory.

Speculative decoding

The DFlash2 draft model(opens in new tab) proposes a block of eight draft tokens per step, and Muse Spark Open verifies them in a single pass. Both server commands above enable it with --speculative-algorithm DFLASH and the three --speculative-* flags after it. Remove those four flags to serve without speculative decoding.

HiCache

HiCache offloads the KV cache to pinned host memory, so evicted prefixes are restored instead of recomputed. Both server commands above enable it with --enable-hierarchical-cache --hicache-size 200.

--hicache-size is the pinned host memory per tensor-parallel rank, in GB: the NVFP4 command (four ranks) pins 800 GB and the BF16 command (eight ranks) pins 1.6 TB. To size the host tier relative to the device KV pool instead, pass --hicache-ratio N. Remove both flags to serve without HiCache.

Disaggregate prefill and decode

Prefill runs MegaMoE and sequence parallelism with HiCache. Decode runs the TensorRT-LLM MoE kernel with DFlash2. Run prefill and decode as two servers on four GPUs each, with a router in front that takes client requests:

B300: both servers share one 8-GPU node, prefill on GPUs 0–3 and decode on GPUs 4–7.
GB300: each server takes a full 4-GPU node, so prefill and decode run on separate nodes.

1. Set the environment for your hardware. Run the matching block before starting each server.

On B300, in the prefill terminal:

bash
export MC_INTRANODE_NVLINK=true
export MC_TCP_ENABLE_CONNECTION_POOL=true
export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=INTRA_NODE_NVLINK
export CUDA_VISIBLE_DEVICES=0,1,2,3

On B300, in the decode terminal:

bash
export MC_INTRANODE_NVLINK=true
export MC_TCP_ENABLE_CONNECTION_POOL=true
export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=INTRA_NODE_NVLINK
export CUDA_VISIBLE_DEVICES=4,5,6,7

On GB300, on both the prefill node and the decode node:

bash
export NCCL_NVLS_ENABLE=1
export NCCL_MNNVL_ENABLE=1
export NCCL_CUMEM_ENABLE=1
export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=NVLINK
export MC_FORCE_MNNVL=1

2. Start the prefill server (MegaMoE, sequence parallelism, and HiCache) on port 30001:

bash
sglang serve \
  --model-path meta-models/Muse-Spark-Open-816B-A42B-NVFP4 \
  --quantization modelopt_fp4 \
  --moe-runner-backend flashinfer_trtllm \
  --tp-size 4 --ep-size 4 \
  --attention-backend trtllm_mha --page-size 128 \
  --mem-fraction-static 0.8 \
  --reasoning-parser muse --tool-call-parser muse \
  --enable-layernorm-sp --enable-megamoe-for-layernorm-sp \
  --enable-hierarchical-cache --hicache-size 200 \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path RadixArk/Muse-Spark-Open-DFlash2 \
  --speculative-draft-attention-backend fa4 \
  --speculative-num-draft-tokens 8 \
  --disaggregation-mode prefill \
  --disaggregation-transfer-backend mooncake \
  --host 0.0.0.0 --port 30001

3. Start the decode server (TensorRT-LLM MoE kernel and DFlash2) on port 30002, in a second terminal on B300 or on the decode node on GB300:

bash
sglang serve \
  --model-path meta-models/Muse-Spark-Open-816B-A42B-NVFP4 \
  --quantization modelopt_fp4 \
  --moe-runner-backend flashinfer_trtllm \
  --tp-size 4 --ep-size 4 \
  --attention-backend trtllm_mha --page-size 128 \
  --mem-fraction-static 0.8 \
  --reasoning-parser muse --tool-call-parser muse \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path RadixArk/Muse-Spark-Open-DFlash2 \
  --speculative-draft-attention-backend fa4 \
  --speculative-num-draft-tokens 8 \
  --disaggregation-mode decode \
  --disaggregation-transfer-backend mooncake \
  --host 0.0.0.0 --port 30002

4. Start the router once both servers are up. Replace <prefill-host> and <decode-host> with addresses the router can reach; on B300, both are 127.0.0.1.

bash
sglang-router launch \
  --pd-disaggregation \
  --prefill http://<prefill-host>:30001 8998 \
  --decode http://<decode-host>:30002 \
  --host 0.0.0.0 --port 8000

8998 after the prefill URL is the prefill server's bootstrap port. Send client requests to the router on port 8000: in the script under Send requests, change the port from 30000 to 8000.

Send requests

Here is a simple script for sending requests:

Python
from openai import OpenAI
client = OpenAI(
    base_url="http://127.0.0.1:30000/v1",
    api_key="null",
    timeout=600.0,
)
model = client.models.list().data[0].id
response = client.chat.completions.create(
    model=model,
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Briefly explain: what is SGLang?"},
    ],
    temperature=1.0,
    top_p=1.0,
    max_tokens=16384, # For setting the maximum number of output tokens
    reasoning_effort="high", # Can be one of low/medium/high/xhigh. xhigh by default.
)
choice = response.choices[0]
print("Reasoning:", getattr(choice.message, "reasoning_content", None))
print("Content:", choice.message.content)
print("Finish reason:", choice.finish_reason)

For other ways to send requests, such as curl or Python requests, see the SGLang tutorial(opens in new tab).

To run Muse Spark Open on AMD GPUs or Intel Xeon, see Hardware platforms. The SGLang cookbook page(opens in new tab) has more deployment options.

For more on SGLang, see the SGLang website(opens in new tab) and the LMSYS blog(opens in new tab).

vLLM

Serve Muse Spark Open with vLLM(opens in new tab) behind an OpenAI-compatible endpoint.

Prerequisites

The downloaded weights. See Get the model.
NVIDIA or AMD GPUs. See Choose a checkpoint.

Install

Prebuilt images are available for both vendors:

NVIDIA: vllm/vllm-openai:muse-spark-open
AMD: vllm/vllm-openai-rocm:muse-spark-open

Use one of the Docker images above to run Muse Spark Open.

Start the server

NVFP4 on 4 × B300, with DSpark speculative decoding:

bash
export VLLM_USE_RUST_FRONTEND=1
vllm serve meta-models/Muse-Spark-Open-816B-A42B-NVFP4 \
    --tensor-parallel-size 4 \
    --enable-expert-parallel \
    --max-model-len 1048576 \
    --reasoning-parser muse \
    --tool-call-parser muse \
    --enable-auto-tool-choice \
    --speculative-config '{"method":"dspark","model": "Inferact/Muse-Spark-Open-DSpark","num_speculative_tokens":7,"draft_sample_method":"probabilistic","rejection_sample_method":"block","enable_adaptive_verification":true}'

For BF16, serve meta-models/Muse-Spark-Open-816B-A42B with --tensor-parallel-size 8 on 8 × B300 and keep the other flags.

--max-model-len 1048576 serves the full 1,048,576-token context window. KV memory on the full-attention layers grows with context length, so if your workload needs less, lower it as described in Context length.

Send a request

vLLM exposes an OpenAI-compatible endpoint on port 8000:

curl
curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "meta-models/Muse-Spark-Open-816B-A42B-NVFP4",
    "messages": [{"role": "user", "content": "Refactor this module."}],
    "reasoning_effort": "xhigh"
  }'

Reasoning effort defaults to xhigh; set reasoning_effort to low, medium, high, or xhigh. See Reasoning strength.

Reasoning is returned separately from content, and tool calls arrive as a standard tool_calls array. Parsing happens server-side, so an OpenAI-compatible harness needs no model-specific handling.

Send image, video, and audio

curl
curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "meta-models/Muse-Spark-Open-816B-A42B-NVFP4",
    "messages": [{"role": "user", "content": [
      {"type": "image_url", "image_url": {"url": "https://example.com/screenshot.png"}},
      {"type": "video_url", "video_url": {"url": "https://example.com/demo.mp4"}},
      {"type": "audio_url", "audio_url": {"url": "https://example.com/clip.wav"}},
      {"type": "text",      "text": "What is happening here, and when?"}
    ]}]
  }'

Stop tokens

Muse Spark Open needs to stop on <|eot|> and the end-of-text token. Never stop on <|eom|>, which ends a message while the turn continues. See Stop tokens.

TensorRT-LLM

NVIDIA TensorRT-LLM(opens in new tab) serves Muse Spark Open on NVIDIA Blackwell GPUs through an OpenAI-compatible server (trtllm-serve). The validated deployment runs the NVFP4 checkpoint on a single 4-GPU Blackwell node (tensor parallel 4, expert parallel 4). The configuration uses an FP8 KV cache, CUDA graphs, chunked prefill, and KV-cache block reuse. The server supports image, video, and audio input, guided decoding (JSON mode and JSON schema), tool calling, and optional speculative decoding.

Install

Muse Spark Open support is in the TensorRT-LLM feat/musespark-open branch. Build TensorRT-LLM from that branch; no pre-built release container or wheel includes Muse Spark Open support.

Clone the branch and build from source following the build-from-source guide(opens in new tab). The only differences from that guide are the branch and the Blackwell CUDA architecture (100-real):

bash
git clone --branch feat/musespark-open https://github.com/NVIDIA/TensorRT-LLM.git
cd TensorRT-LLM
git lfs pull

Then, inside the TensorRT-LLM development container (see the guide for the devel container tag and the required docker run flags), build and install in editable mode, targeting Blackwell:

bash
python3 scripts/build_wheel.py --use_ccache -a "100-real" \
    --skip_building_wheel --linking_install_binary
pip install -e .

You also need a complete Hugging Face-format Muse Spark Open checkpoint with its tokenizer and processor files: meta-models/Muse-Spark-Open-816B-A42B-NVFP4(opens in new tab) (a BF16 variant, meta-models/Muse-Spark-Open-816B-A42B(opens in new tab), is also published). The NVFP4 checkpoint ships the routed-expert weights pre-quantized to NVFP4; no additional quantization step is needed.

Start the server

Muse Spark Open at TP4/EP4 fits on a single Blackwell 4-GPU node, so trtllm-serve launches directly (no multi-node launcher needed). Write the recommended settings (next section) to a YAML file and pass it with --config:

bash
MODEL=meta-models/Muse-Spark-Open-816B-A42B-NVFP4   # or a local checkpoint path
trtllm-serve ${MODEL} \
    --host 0.0.0.0 --port 8000 \
    --config serve_config.yaml \
    --tool_parser musespark \
    --reasoning_parser musespark

Pass --tool_parser musespark and --reasoning_parser musespark explicitly: they enable tool-call and reasoning-content extraction on /v1/chat/completions. The server is ready when the log shows Application startup complete; the first query may take longer due to warmup.

Recommended settings

Start from the validated aggregated serving configuration, serve_config.yaml, and tune it for your workload:

YAML
tensor_parallel_size: 4
moe_expert_parallel_size: 4
enable_attention_dp: false
max_batch_size: 256
max_num_tokens: 8192
max_seq_len: 49152
trust_remote_code: true
enable_chunked_prefill: true
disable_overlap_scheduler: false
attn_backend: TRTLLM
allreduce_strategy: NCCL
guided_decoding_backend: xgrammar
cuda_graph_config:
  max_batch_size: 256
  enable_padding: true
moe_config:
  backend: TRTLLM
kv_cache_config:
  # FP8 KV cache is required on GB300 (SM103) when chunked prefill or
  # block reuse is enabled.
  dtype: fp8
  enable_block_reuse: true
  enable_partial_reuse: true
  free_gpu_memory_fraction: 0.8

Notes:

FP8 KV cache is required on GB300 (SM103) with chunked prefill or KV reuse. For this model's head dimension (64), the fused paged-context attention kernel exists only for the FP8 KV-cache dtype; a 16-bit KV cache in that combination is refused at startup.
max_seq_len: 49152 is the validated serving window; the model supports up to 1M tokens of context. Longer windows need proportionally more KV-cache memory.
On out-of-memory errors, reduce kv_cache_config.free_gpu_memory_fraction, max_batch_size, or max_seq_len.

Send requests

The server exposes the OpenAI-compatible API:

curl
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "meta-models/Muse-Spark-Open-816B-A42B-NVFP4",
    "messages": [{"role": "user", "content": "Where is New York?"}],
    "max_tokens": 16384
}'

Muse Spark Open is a reasoning model: with --reasoning_parser musespark, each chat response returns the chain-of-thought in a separate reasoning_content field and the user-facing answer in content. Both are drawn from the same max_tokens budget, so set max_tokens large enough to cover the reasoning and the answer. Too small a budget is spent entirely on reasoning and returns an empty content with "finish_reason": "length".

Guided decoding (JSON mode and JSON schema via the xgrammar backend) works through the standard response_format field. The constraint applies to content (the answer), so the budget must also leave room for the model to finish reasoning first:

curl
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "meta-models/Muse-Spark-Open-816B-A42B-NVFP4",
    "messages": [{"role": "user", "content": "Give me a city and its country as JSON."}],
    "max_tokens": 16384,
    "response_format": {"type": "json_object"}
}'

Image, video, and audio inputs are accepted through the standard multimodal chat-completions content parts; multimodal token budgets count against max_num_tokens and max_seq_len like text tokens.

The TensorRT-LLM deployment guide for Muse Spark Open(opens in new tab) covers key YAML options, multimodal token budgets, speculative decoding, benchmarking with tensorrt_llm.serve.scripts.benchmark_serving, and troubleshooting.
For disaggregated serving, with prefill and decode on separate workers, see NVIDIA Dynamo, which uses TensorRT-LLM as its backend engine.

NVIDIA Dynamo

NVIDIA Dynamo(opens in new tab) provides an OpenAI-compatible frontend, reasoning and tool-call parsing, disaggregated serving, and KV-aware routing.

Install

See the Dynamo installation guide(opens in new tab). Use the Muse Spark Open-enabled image and checkpoint specified in the model recipe, including the latest tokenizer, processor, and chat template.

Start the server

Follow the model recipe for launch commands and backend settings. The Dynamo quickstart(opens in new tab) explains how the frontend and model worker run together.

Recommended settings

See the Dynamo recipes(opens in new tab) for recommended deployment settings.

Send requests

See the request examples in the Dynamo quickstart(opens in new tab) and model recipe. Chat completions separate reasoning_content from content and return structured tool_calls when tools are supplied.

Hugging Face Transformers

Muse Spark Open comes with a native Hugging Face Transformers implementation, so you can run it using familiar APIs and integrate it into your workflows. It supports text, images, audio, and video, as well as agentic use cases including tool calls. Checkpoints are available in BF16 and NVFP4 formats. You can post-train Muse Spark Open with Transformer Reinforcement Learning (TRL) to adapt it to your own tasks and datasets; see Post-train with TRL.

Install

Install Hugging Face Transformers with:

bash
pip install -U "transformers>=5.19"

See the Transformers installation guide(opens in new tab) for more options.

Run the model

The Muse Spark Open implementation provides MuseSparkForConditionalGeneration for text generation from multimodal inputs and MuseSparkProcessor for preparing conversations. The processor combines the tokenizer, image and video processors, audio feature extractor, and chat template.

Use a DistributedConfig to configure the parallelism settings. tp_size is the number of devices across which the dense layers are sharded, and ep_size is the number of devices across which the experts are distributed.

The example below shows how to define the distributed config, load the model with AutoModelForMultimodalLM, preprocess the inputs, and generate and decode the output.

Python
import os
from transformers import AutoModelForMultimodalLM, AutoProcessor, DistributedConfig
distributed_config = DistributedConfig(
    tp_size=int(os.environ["WORLD_SIZE"]),
    ep_size=int(os.environ["WORLD_SIZE"]),
)
processor = AutoProcessor.from_pretrained("meta-models/Muse-Spark-Open-816B-A42B")
model = AutoModelForMultimodalLM.from_pretrained(
    "meta-models/Muse-Spark-Open-816B-A42B",
    distributed_config=distributed_config,
)
messages = [
    {"role": "user", "content": "Explain in one sentence why the sky is usually blue."},
]
inputs = processor.apply_chat_template(
    messages,
    reasoning_effort="low",  # default: xhigh, options: low/medium/high/xhigh
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device).to(model.dtype)
input_len = inputs["input_ids"].shape[-1]
outputs = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))
# to=self<|message|>We need to explain in one sentence why sky is usually blue.
# ...
# <|eom|><|start|>assistant to=user<|message|>The sky appears blue because air molecules scatter shorter-wavelength blue light from the sun more strongly than longer-wavelength red light.<|eot|>

Save the script above as run_muse_spark_open.py and launch it with torchrun on every node using Slurm:

bash
#!/bin/bash
#SBATCH --job-name=muse-spark-open
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=1
#SBATCH --gpus-per-node=8
export MASTER_ADDR=$(scontrol show hostnames "$SLURM_JOB_NODELIST" | head -n 1)
export MASTER_PORT=29500
srun bash -c 'torchrun \
    --nnodes=$SLURM_NNODES \
    --nproc-per-node=8 \
    --node-rank=$SLURM_NODEID \
    --master-addr=$MASTER_ADDR \
    --master-port=$MASTER_PORT \
    run_muse_spark_open.py'

NVFP4 weights

NVFP4 quantized weights(opens in new tab) allow running the model with lower memory requirements on compatible hardware. The inference code is the same apart from the model ID, but NVFP4 needs a Transformers build from a pull request branch, installed below.

NVFP4 support requires Transformers from the NVFP4 pull request branch(opens in new tab):

bash
pip install git+https://github.com/huggingface/transformers.git@finegrained-per-scheme-demo

The weights can then be loaded by setting the model name to "meta-models/Muse-Spark-Open-816B-A42B-NVFP4" as demonstrated below.

Python
import os
from transformers import AutoModelForMultimodalLM, AutoProcessor, DistributedConfig
distributed_config = DistributedConfig(
    tp_size=int(os.environ["WORLD_SIZE"]),
    ep_size=int(os.environ["WORLD_SIZE"]),
)
processor = AutoProcessor.from_pretrained("meta-models/Muse-Spark-Open-816B-A42B-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained(
    "meta-models/Muse-Spark-Open-816B-A42B-NVFP4",
    distributed_config=distributed_config,
)
messages = [
    {"role": "user", "content": "Explain in one sentence why the sky is usually blue."},
]
inputs = processor.apply_chat_template(
    messages,
    reasoning_effort="low",
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device).to(model.dtype)
input_len = inputs["input_ids"].shape[-1]
outputs = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))

Launch it with torchrun and the same Slurm script as Run the model.

See the full Transformers documentation for Muse Spark(opens in new tab) to see how to use the pipeline API and run inference for all modalities, including image, video, and audio.

The Hugging Face blog post on Muse Spark Open(opens in new tab) covers the model in depth and more ways to deploy it with Transformers.

llama.cpp

Portable inference across Metal, CUDA, ROCm, Vulkan, and CPU. llama.cpp and the Llama app support Muse Spark Open.

Install

Install a prebuilt llama binary for your platform:

bash
curl -LsSf https://llama.app/install.sh | sh

Serve

llama serve -hf downloads, caches, and serves a model in one command. Muse Spark Open supports up to 1,048,576 tokens of context.

bash
llama serve -hf ggml-org/Muse-Spark-Open-816B-A42B-GGUF --chat-template-kwargs '{"reasoning_effort":"low"}'

To serve a specific quantization, add its suffix. ggml-org ships three quantizations for this release: Q4_K_S, Q8_0, and Q2_K.

bash
llama serve -hf ggml-org/Muse-Spark-Open-816B-A42B-GGUF:Q4_K_S
llama serve -hf ggml-org/Muse-Spark-Open-816B-A42B-GGUF:Q8_0
llama serve -hf ggml-org/Muse-Spark-Open-816B-A42B-GGUF:Q2_K

The server exposes an OpenAI-compatible endpoint. Open http://localhost:8080 to chat with Muse Spark Open in the Llama WebUI, which supports images, audio, and Model Context Protocol (MCP) tools.

Useful flags:

Flag	Why
`-a muse-spark-open`	API alias. By default, this is the checkpoint path.
`--chat-template-kwargs`	Sets `reasoning_effort`, see below. Template default is `xhigh`.
`-np N`	Concurrent slots. The context length is divided across the slots, so long-context agents should use `-np 1`.
`--api-key`	Optional key to gate endpoints. `/health` and `/v1/models` still answer without it.

Smoke test (--noproxy ensures the loopback request connects directly):

curl
curl -s --noproxy '*' http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer <your-key>" \
  -d '{"model":"muse-spark-open","messages":[{"role":"user","content":"What is 17 * 23?"}],"temperature":1.0}'

Control reasoning

The chat template reads reasoning_effort and defaults to xhigh.

Set it server-wide with --chat-template-kwargs, or per request:

JSON
{"model":"muse-spark-open","messages":[...],"chat_template_kwargs":{"reasoning_effort":"low"}}

The model supports four levels: low, medium, high, xhigh.

Reasoning can't be turned off. The chat template rejects "reasoning_effort": "none", and --reasoning off leaves the output unchanged. Set reasoning_effort to low to spend fewer thinking tokens.

Vision

You can upload images through the Web UI. For applications that use the server endpoint, send a remote URL, a data:image/...;base64,... URI, or a local path:

curl
curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer <your-key>" \
  -d '{"model":"muse-spark-open","messages":[{"role":"user","content":[
        {"type":"text","text":"What is in this image?"},
        {"type":"image_url","image_url":{"url":"data:image/png;base64,..."}}
      ]}],"temperature":1.0}'

Audio

Similarly to vision, you can either upload through the Web UI or send base64 encoded files to the endpoint. Audio follows the OpenAI input_audio shape: raw base64 in data, not a data URL, and the file type (wav or mp3) in format.

curl
curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer <your-key>" \
  -d '{"model":"muse-spark-open","messages":[{"role":"user","content":[
        {"type":"text","text":"Transcribe this audio."},
        {"type":"input_audio","input_audio":{"data":"<base64>","format":"wav"}}
      ]}],"temperature":1.0}'

CLI

You can also use llama cli to interact with the model through CLI.

bash
llama cli -hf ggml-org/Muse-Spark-Open-816B-A42B-GGUF -c 32768

After you launch llama cli, use the /image command to pass in images. You can then enter a text prompt.

bash
> /image image.png
Loaded media from 'image.png'
> describe this image

Similarly, to pass audio:

bash
> /audio /path/example_audio.mp3
Loaded media from '/path/example_audio.mp3'
> transcribe this audio

Verify tool calling

curl
curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer <your-key>" \
  -d '{"model":"muse-spark-open","messages":[{"role":"user","content":"What is the weather in Paris in celsius? Use the tool."}],
       "tools":[{"type":"function","function":{
         "name":"get_weather","description":"Get current weather for a city.",
         "parameters":{"type":"object","properties":{
           "city":{"type":"string"},"units":{"type":"string","enum":["celsius","fahrenheit"]}},
           "required":["city"]}}}],
       "tool_choice":"auto","temperature":1.0}'

The command above returns finish_reason: "tool_calls", a tool_calls array containing get_weather(city="Paris", units="celsius"), and reasoning in reasoning_content. Tool-call parsing occurs server-side, so OpenAI-compatible clients require no Muse Spark Open–specific logic.

Context per slot

Check n_ctx_slot in the startup log to determine the per-generation context limit. Don't read the limit from -c: it sets the total context length, shared across slots. When a generation exceeds its slot limit, the server may return neither an answer nor an error, making batch and evaluation failures difficult to diagnose.

To increase concurrency while retaining the same per-slot limit, increase -c in proportion to -np:

bash
  -c 524288 -np 2        # 262144 per slot and 2-way concurrent

Unsloth

Run Muse Spark Open locally with Unsloth Desktop and Unsloth Dynamic GGUFs. Use the desktop interface to download and chat with the model, or serve an authenticated API for your applications and coding agents. Unsloth runs on macOS, Windows, Linux, and WSL, with CPU and GPU inference powered by llama.cpp.

Muse Spark Open is Meta's 816B-parameter mixture-of-experts model with about 42B active parameters, built for multimodal reasoning, coding, and agentic workflows. It supports text, image, video, and audio input with text output. This recipe covers text chat and API serving; multimodal input depends on the inference backend and the model assets you load.

Choose a quantization

Start with a Dynamic 2-bit GGUF and choose a variant that fits your hardware. Unsloth's guide gives the following approximate memory planning ranges: system RAM plus VRAM, or unified memory on Apple silicon. Requirements vary with the exact quantization, context length, KV-cache format, and runtime configuration.

Quantization	Approximate total memory
2-bit	300-315 GB
3-bit	365-390 GB
4-bit	470-500 GB
6-bit	670-700 GB
8-bit	870-950 GB

BF16 language-model weights alone require approximately 1.63 TB before runtime overhead. Leave memory beyond the downloaded model size for the operating system, runtime, and KV cache. Longer contexts increase memory requirements. Offloading layers to system RAM reduces VRAM requirements; if memory pressure causes disk paging, generation can become much slower.

Install Unsloth Desktop

Download Unsloth(opens in new tab) and complete the installer for your platform:

Alternatively, install from a terminal:

bash
curl -fsSL https://unsloth.ai/install.sh | sh

powershell
irm https://unsloth.ai/install.ps1 | iex

Use a recent Unsloth release with Muse Spark Open support. See the platform installation guides(opens in new tab) for additional setup instructions.

Start the server

In Unsloth Desktop:

Launch Unsloth from your installed app shortcut.
Open Model Hub, search for "Muse Spark Open", and select the Unsloth GGUF repository linked above.
Download a quantization that fits your available memory. Start with Dynamic 2-bit; the CLI example below selects UD-Q2_K_XL.
Load the model from Select model on the Chat page and send a short message to confirm that it responds.
Open Settings > API, create an API key, and copy it. Use the endpoint URL shown in Unsloth when configuring your client.

Unsloth provides automatic per-model inference presets and lets you edit sampling settings, context length, and the chat template. Check the values against the recommended settings below when loading Muse Spark Open.

To load the model and start its API from the command line with the Unsloth CLI:

bash
unsloth run \
  --model unsloth/Muse-Spark-Open-816B-A42B-GGUF:UD-Q2_K_XL \
  -H 127.0.0.1 \
  -p 8888 \
  --context-length 32768 \
  --temperature 1.0 \
  --top-p 1.0 \
  --min-p 0.0 \
  --reasoning-effort medium

The :UD-Q2_K_XL suffix selects the quantization. Unsloth starts the server, loads the UI, and prints the endpoint URL and API key. Keep this terminal running while sending requests.

This example starts with a 32,768-token context to limit KV-cache memory. Increase --context-length as your available memory allows. The medium reasoning level is a starting choice for interactive use; change it to another supported level as needed.

For a larger-memory configuration, select :UD-Q4_K_XL instead. Additional llama-server runtime flags let you adjust GPU layers, CPU threads, sampling, and networking; see the Unsloth API guide(opens in new tab).

Recommended settings

Setting	Recommended value
Temperature	`1.0`
Top-p	`1.0`
Min-p	`0.0` in the CLI example
Starting quantization	Dynamic 2-bit
Example serving context	`32768` tokens; adjust for available memory
Model maximum context	`1048576` tokens
Supported reasoning effort	`low`, `medium`, `high`, `xhigh`

Use --reasoning-effort to pass the selected effort level to the model's chat template. The level takes effect only when the loaded template and backend support it. Choose low or medium for lighter reasoning, and high or xhigh for more demanding tasks. Muse Spark Open evaluations use xhigh for maximum reasoning strength.

The model's maximum context is not a recommended allocation for every machine. Context includes the input and generated tokens, and a larger KV cache needs additional memory. Allow enough output tokens for both reasoning and the final answer.

Send requests

Unsloth exposes OpenAI-compatible /v1/chat/completions and /v1/responses endpoints, plus the Anthropic-compatible /v1/messages endpoint. Authenticate with your Unsloth API key. The examples below use Chat Completions.

In a second terminal, set the base URL and the API key copied from Unsloth. These shell examples use Bash on macOS, Linux, or WSL, and require Python 3 for model discovery.

bash
export UNSLOTH_BASE_URL="http://127.0.0.1:8888"
export UNSLOTH_API_KEY="paste-your-unsloth-api-key-here"

Use the URL printed by your server if it differs. The /v1/models endpoint lists loaded and locally cached models. Select the loaded Muse Spark Open model's exact ID:

bash
UNSLOTH_MODEL_ID="$(
  curl --fail-with-body "${UNSLOTH_BASE_URL}/v1/models" \
    -H "Authorization: Bearer ${UNSLOTH_API_KEY}" |
    python3 -c '
import json
import sys
models = [
    model
    for model in json.load(sys.stdin)["data"]
    if model.get("loaded") is True
    and "muse-spark-open" in model["id"].lower()
]
if len(models) != 1:
    raise SystemExit("Load exactly one Muse Spark Open model and retry.")
print(models[0]["id"])
'
)" && export UNSLOTH_MODEL_ID

Send a chat request. The Python example needs the OpenAI SDK (python3 -m pip install -U openai) and runs in the same environment as the exported client variables:

bash
curl --fail-with-body "${UNSLOTH_BASE_URL}/v1/chat/completions" \
  -H "Authorization: Bearer ${UNSLOTH_API_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<JSON
{
  "model": "${UNSLOTH_MODEL_ID}",
  "messages": [
    {"role": "user", "content": "Explain mixture-of-experts models in three sentences."}
  ],
  "temperature": 1.0,
  "top_p": 1.0,
  "min_p": 0.0,
  "reasoning_effort": "medium",
  "enable_tools": false,
  "max_tokens": 8192,
  "stream": false
}
JSON

python
import os
from openai import OpenAI
client = OpenAI(
    base_url=os.environ["UNSLOTH_BASE_URL"].rstrip("/") + "/v1",
    api_key=os.environ["UNSLOTH_API_KEY"],
)
response = client.chat.completions.create(
    model=os.environ["UNSLOTH_MODEL_ID"],
    messages=[
        {"role": "user", "content": "Explain mixture-of-experts models in three sentences."}
    ],
    temperature=1.0,
    top_p=1.0,
    reasoning_effort="medium",
    max_tokens=8192,
    extra_body={"min_p": 0.0, "enable_tools": False},
)
print(response.choices[0].message.content)

min_p and enable_tools are Unsloth extensions to the Chat Completions format. Sending sampling values explicitly also applies them when using the desktop workflow. enable_tools: false keeps this request as text generation. For streaming, set "stream": true and add --no-buffer to curl.

Check function calling

Send an OpenAI-style tool definition to request a structured function call:

curl
curl --fail-with-body "${UNSLOTH_BASE_URL}/v1/chat/completions" \
  -H "Authorization: Bearer ${UNSLOTH_API_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<JSON
{
  "model": "${UNSLOTH_MODEL_ID}",
  "messages": [
    {"role": "user", "content": "Use add_integers to calculate 17 plus 25."}
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "add_integers",
        "description": "Add two integers.",
        "parameters": {
          "type": "object",
          "properties": {
            "a": {"type": "integer"},
            "b": {"type": "integer"}
          },
          "required": ["a", "b"]
        }
      }
    }
  ],
  "tool_choice": "required",
  "temperature": 1.0,
  "top_p": 1.0,
  "min_p": 0.0,
  "reasoning_effort": "medium",
  "enable_tools": false,
  "max_tokens": 8192,
  "stream": false
}
JSON

Unsloth supports this client-side function-calling format; successful tool use also depends on the loaded model and its chat template. Check for a structured choices[0].message.tool_calls array. Your application executes client-defined functions: parse the arguments, run the function, then send the assistant's tool-call message and a role: "tool" result with the matching tool_call_id in the next request. This request disables Unsloth's built-in server-side tools so your application handles the defined function.

See the curl and HTTP guide(opens in new tab) and Python SDK guide(opens in new tab) for additional endpoint examples.

Connect a coding agent

Install your coding agent if it is not already available, load Muse Spark Open in Unsloth, and connect the agent:

bash
unsloth start claude

You can also select the model while launching the agent:

bash
unsloth start claude \
  --model unsloth/Muse-Spark-Open-816B-A42B-GGUF:UD-Q2_K_XL \
  --context-length 32768 \
  --temperature 1.0 \
  --top-p 1.0 \
  --min-p 0.0 \
  --reasoning-effort medium

On the default local address, passing --model starts a temporary server if Unsloth is not already running; that server stops when the agent exits. If Unsloth is already running, the launcher connects to it. Use --temperature for the Unsloth agent launcher. See Unsloth Start(opens in new tab) for supported agents and configuration options.

Explore other workflows

Unsloth inference guide(opens in new tab): chat settings, web search, and Python or Bash execution.
Unsloth Desktop(opens in new tab): model downloads, training, and exporting workflows for supported models.
MCP integrations(opens in new tab): connect additional tools.
llama.cpp(opens in new tab): run the GGUFs directly. Use a current build with Muse Spark Open support. Enable CUDA for NVIDIA GPUs; on Apple silicon, disable CUDA and use the default Metal backend.
llama-server documentation(opens in new tab): context sizing, GPU offload, threading, and sampling flags.

MLX

Run Muse Spark Open on Apple silicon with MLX and the NVFP4 checkpoint. The NVFP4 weights need about 493 GB of memory before any inference overhead, so the model doesn't fit on one Mac. Serve it across two Mac Studio computers with M5 Ultra and 512 GB of unified memory each, or four with 256 GB each. Four Macs serve fastest.

Prerequisites

macOS 26.2 or later on every Mac. It adds the RDMA-over-Thunderbolt driver that MLX's jaccl backend uses. Turn on RDMA on each Mac as described in the MLX distributed documentation(opens in new tab).
MLX 0.30.1 or later.
A direct Thunderbolt 5 link between every pair of Macs. Connect each pair with two cables to double the link bandwidth. With four Macs, every Mac connects to each of the other three.
Passwordless SSH from the primary Mac to every other Mac. These instructions call the primary mac-studio-1.local and the others mac-studio-2.local and so on; adjust the names to match your machines.

Check that the primary can reach each other Mac without a password prompt:

bash
ssh mac-studio-2.local 'echo ok'

If that prompts for a password, set up key-based access first:

bash
ssh-keygen -t ed25519
ssh-copy-id mac-studio-2.local

Install

mlx.launch starts the program over SSH on every Mac, so install mlx-lm on each one:

bash
pip install -U mlx-lm

Each Mac loads the model from its own disk, so each needs room for the NVFP4 download, about 537 GB.

Create the hostfile

mlx.launch reads which Macs to use from a hostfile. List every Mac in it, two or four. For the jaccl backend, each entry also carries the host's Thunderbolt IP addresses and RDMA devices. The MLX distributed documentation(opens in new tab) describes the format, and mlx.distributed_config can generate the file. These instructions save it as hostfile.json on the primary.

Start the server

Run this on the primary. It starts mlx_lm.server on every Mac in the hostfile and serves an OpenAI-compatible API on port 8080:

bash
mlx.launch --hostfile hostfile.json --backend jaccl --env MLX_METAL_FAST_SYNCH=1 -- \
  mlx_lm.server --model meta-models/Muse-Spark-Open-816B-A42B-NVFP4 --port 8080

Everything after -- is the program that mlx.launch runs on each Mac. --backend jaccl selects the RDMA-over-Thunderbolt backend. --env MLX_METAL_FAST_SYNCH=1 turns on an experimental MLX feature for fast CPU-GPU synchronization, which cuts communication overhead between the CPU, which drives the Thunderbolt link, and the GPU, which does the compute.

Generate from the command line

To test without a server, run mlx_lm.generate through the same launcher. The chat template defaults to xhigh reasoning, so leave room for reasoning tokens before the answer, and pass the recommended sampling settings:

bash
mlx.launch --hostfile hostfile.json --backend jaccl --env MLX_METAL_FAST_SYNCH=1 -- \
  mlx_lm.generate \
    --model meta-models/Muse-Spark-Open-816B-A42B-NVFP4 \
    --prompt "Give me a one-sentence summary of what you can do." \
    --max-tokens 4096 \
    --temp 1.0 \
    --top-p 1.0

Hosted providers

Call Muse Spark Open through a hosted provider when you'd rather not provision GPU capacity yourself. These providers serve the open weights:

Provider	Model string	Get a key
Together AI	`meta-models/muse-spark-open`	Together AI API keys(opens in new tab)
Fireworks AI	`accounts/fireworks/models/muse-spark-open`	API keys page(opens in new tab)
Baseten	`meta-models/muse-spark-open`	Baseten app(opens in new tab), Settings → API Keys
Groq	`meta-models/muse-spark-open`	GroqCloud console(opens in new tab), once your organization has access
Nebius Token Factory	`meta-models/muse-spark-open`	Token Factory API keys(opens in new tab)

You can also call Muse Spark Open through Modal and OpenRouter.

Send a request

Pick your provider's tab:

bash
curl https://api.together.xyz/v1/chat/completions \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-models/muse-spark-open",
    "messages": [{"role": "user", "content": "Explain mixture-of-experts models in two sentences."}]
  }'

bash
curl https://api.fireworks.ai/inference/v1/chat/completions \
  -H "Authorization: Bearer $FIREWORKS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "accounts/fireworks/models/muse-spark-open",
    "messages": [{"role": "user", "content": "Explain mixture-of-experts models in two sentences."}],
    "max_tokens": 4096
  }'

python
import os
from openai import OpenAI
client = OpenAI(
    api_key=os.environ["BASETEN_API_KEY"],
    base_url="https://inference.baseten.co/v1",
)
response = client.chat.completions.create(
    model="meta-models/muse-spark-open",
    messages=[{"role": "user", "content": "Explain what a stack trace is in two sentences."}],
)
print(response.choices[0].message.content)

bash
# GroqCloud access requires approval for your organization; see Groq below.
curl https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-models/muse-spark-open",
    "reasoning_effort": "medium",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'

bash
curl https://api.tokenfactory.us-north1.nebius.com/v1/chat/completions \
  -H "Authorization: Bearer $NEBIUS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-models/muse-spark-open",
    "messages": [{"role": "user", "content": "Say hello in one sentence."}],
    "max_tokens": 16384
  }'

Together AI

Together AI serves Muse Spark Open on serverless and on-demand dedicated endpoints through an OpenAI-compatible API.

Model string

Text
meta-models/muse-spark-open

Get a key

Create an API key(opens in new tab) and export it:

bash
export TOGETHER_API_KEY="YOUR_API_KEY"

Send requests

The base URL is https://api.together.xyz/v1. Any OpenAI-compatible client works.

curl
curl https://api.together.xyz/v1/chat/completions \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-models/muse-spark-open",
    "messages": [{"role": "user", "content": "Explain mixture-of-experts models in two sentences."}]
  }'

Fireworks AI

Fireworks serves Muse Spark Open on serverless inference. You pay per token and send requests to an OpenAI-compatible endpoint. There is no deployment to provision. The model accepts text and images, supports tool calling, and has a context window of 1,048,576 tokens. You can set how much the model reasons on each request with reasoning_effort.

For dedicated capacity, guaranteed throughput, or private networking, run the same model on an on-demand deployment.

Model string

Text
accounts/fireworks/models/muse-spark-open

Get a key

Create a Fireworks account at fireworks.ai(opens in new tab).
Create an API key on the API keys page(opens in new tab), or run firectl api-key create.
Export the key:

bash
export FIREWORKS_API_KEY="fw_..."

Send requests

The base URL is https://api.fireworks.ai/inference/v1. Any OpenAI-compatible client works.

curl
curl https://api.fireworks.ai/inference/v1/chat/completions \
  -H "Authorization: Bearer $FIREWORKS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "accounts/fireworks/models/muse-spark-open",
    "messages": [{"role": "user", "content": "Explain mixture-of-experts models in two sentences."}],
    "max_tokens": 4096
  }'

With the OpenAI Python SDK:

bash
pip install openai

Python
import os
from openai import OpenAI
client = OpenAI(
    api_key=os.environ["FIREWORKS_API_KEY"],
    base_url="https://api.fireworks.ai/inference/v1",
)
response = client.chat.completions.create(
    model="accounts/fireworks/models/muse-spark-open",
    messages=[{"role": "user", "content": "Explain mixture-of-experts models in two sentences."}],
    max_tokens=4096,
)
print(response.choices[0].message.content)

Reasoning effort. Muse Spark Open always reasons. Set reasoning_effort to low, medium, high, or xhigh. The default is xhigh. Higher settings think longer and cost more output tokens. Use xhigh for hard reasoning, coding, and agentic tasks. Reasoning is returned in reasoning_content, separate from content. Thinking cannot be turned off, so reasoning_effort="none" is rejected.

Python
# Uses the client from the first Python example in this section
response = client.chat.completions.create(
    model="accounts/fireworks/models/muse-spark-open",
    messages=[{"role": "user", "content": "How many primes are there below 1000?"}],
    reasoning_effort="xhigh",
    max_tokens=32768,
)

Images. Pass image URLs or base64 data URLs in the message content.

Python
# Uses the client from the first Python example in this section
response = client.chat.completions.create(
    model="accounts/fireworks/models/muse-spark-open",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What is in this image?"},
            {"type": "image_url", "image_url": {"url": "http://images.cocodataset.org/val2017/000000039769.jpg"}},
        ],
    }],
)

Tool calling. Pass tools in the OpenAI function-calling format. Tool calls come back in message.tool_calls.

Python
# Uses the client from the first Python example in this section
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]
response = client.chat.completions.create(
    model="accounts/fireworks/models/muse-spark-open",
    messages=[{"role": "user", "content": "What's the weather in Menlo Park?"}],
    tools=tools,
)
print(response.choices[0].message.tool_calls)

Streaming (stream=True) and JSON mode (response_format={"type": "json_object"}) work the same way they do for other Fireworks chat models.

Serverless is billed per token for input, cached input, and output. Current rates are on the model page(opens in new tab).
Prompt caching is automatic. Repeated prefixes, such as long system prompts and agent histories, are billed at the cached-input rate.
For dedicated GPUs, higher rate limits, or reserved throughput, create an on-demand deployment(opens in new tab) of the same model, or contact Fireworks(opens in new tab).
Fireworks also serves an Anthropic-compatible Messages API(opens in new tab), so Muse Spark Open can run in coding agents built for that API.

Baseten

Baseten serves Muse Spark Open on Model APIs (serverless, pay per token) and Dedicated Inference (single-tenant capacity), with the full 1,048,576-token context window and text, image, video, and audio input.

Model string

Text
meta-models/muse-spark-open

Get a key

Create an account in the Baseten app(opens in new tab).
Go to Settings → API Keys and create a key.

Send requests

The endpoint is OpenAI-compatible. Point any OpenAI client at Baseten:

Python
import os
from openai import OpenAI
client = OpenAI(
    api_key=os.environ["BASETEN_API_KEY"],
    base_url="https://inference.baseten.co/v1",
)
response = client.chat.completions.create(
    model="meta-models/muse-spark-open",
    messages=[{"role": "user", "content": "Explain what a stack trace is in two sentences."}],
)
print(response.choices[0].message.content)

Image, video, and audio inputs use the standard OpenAI content-part format (image_url, video_url, audio_url). Send media larger than ~74 MB by URL rather than inline base64; video inputs by URL are supported up to 200 MB.

Pricing: see the Baseten model page for Muse Spark Open(opens in new tab).
The video encoder samples at most 96 frames per clip: a 720p clip costs about 144 tokens per second, capped at ~6.9K tokens from about 48 seconds on, so longer clips (up to 300 s) cost no more than that.

Modal serves Muse Spark Open. See Muse Spark Open on Modal(opens in new tab) for setup and request examples, and the Modal blog post on Muse Spark Open(opens in new tab).

OpenRouter

OpenRouter routes requests to upstream providers rather than hosting the model, so the provider serving your request depends on your routing configuration. See Muse Spark Open on OpenRouter(opens in new tab) for the model string and setup.

Groq

GroqCloud serves Muse Spark Open through an OpenAI-compatible API. Access requires approval for your organization; to request it, contact the Groq team(opens in new tab).

Model string

Text
meta-models/muse-spark-open

On GroqCloud the model has a 131,072-token context window and supports tool calling, structured outputs, and image input.

Get a key

Once your organization has access, create an API key in the GroqCloud console(opens in new tab) and export it:

bash
export GROQ_API_KEY="YOUR_API_KEY"

Send requests

The base URL is https://api.groq.com/openai/v1. Use a Groq SDK, or any OpenAI SDK with base_url set to that URL.

curl
curl https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-models/muse-spark-open",
    "reasoning_effort": "medium",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'

reasoning_effort accepts low, medium, high, or xhigh, and defaults to xhigh. Lower settings respond faster. See Reasoning strength.

For access questions, contact Groq sales(opens in new tab).

Nebius Token Factory

Nebius Token Factory serves Muse Spark Open on a serverless endpoint with an OpenAI-compatible API. You pay per token. To send a request, you need an API key and the model ID below.

Model string

The model ID is case-sensitive:

Text
meta-models/muse-spark-open

Set your client's base URL to:

Text
https://api.tokenfactory.us-north1.nebius.com/v1/

Get a key

Create a Token Factory account(opens in new tab) or sign in.
Complete billing setup(opens in new tab). New accounts need a bank card.
Select your project and open API keys(opens in new tab).
Create a key and copy it. You can't view it again later.
Export the key:

bash
export NEBIUS_API_KEY="YOUR_API_KEY"

Keep the key out of source code and browser applications. Check that the key can list models:

bash
curl --fail-with-body --silent --show-error \
  https://api.tokenfactory.us-north1.nebius.com/v1/models \
  -H "Authorization: Bearer $NEBIUS_API_KEY"

Look for meta-models/muse-spark-open in the response. If it's missing, check model availability for the project that owns your key.

Send requests

curl
curl --fail-with-body --silent --show-error \
  https://api.tokenfactory.us-north1.nebius.com/v1/chat/completions \
  -H "Authorization: Bearer $NEBIUS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-models/muse-spark-open",
    "messages": [
      {"role": "user", "content": "Say hello in one sentence."}
    ],
    "max_tokens": 16384
  }'

The answer is in choices[0].message.content.

With the OpenAI Python SDK:

bash
pip install openai

Python
import os
from openai import OpenAI
client = OpenAI(
    api_key=os.environ["NEBIUS_API_KEY"],
    base_url="https://api.tokenfactory.us-north1.nebius.com/v1/",
)
response = client.chat.completions.create(
    model="meta-models/muse-spark-open",
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
    max_tokens=16384,
)
print(response.choices[0].message.content)

Tool calling. Pass tools in the OpenAI function-calling format. This request asks for a get_weather call with a city argument:

Python
# Uses the client from the first Python example in this section
response = client.chat.completions.create(
    model="meta-models/muse-spark-open",
    messages=[{"role": "user", "content": "What is the weather in Paris?"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city.",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
                "additionalProperties": False,
            },
        },
    }],
    tool_choice={"type": "function", "function": {"name": "get_weather"}},
    max_tokens=16384,
)
for call in response.choices[0].message.tool_calls or []:
    print(call.id, call.function.name, call.function.arguments)

The request doesn't fetch the weather itself. Your application runs the function and sends the result back with the returned call ID. See the Chat Completions reference(opens in new tab) for the tools and tool-message fields.

Token Factory quickstart(opens in new tab) covers account setup and your first request.
API documentation(opens in new tab) lists endpoints, request parameters, and response formats.
Token Factory Cookbook(opens in new tab) has reusable code examples and sample applications.

Next steps

Once your server is up, set the sampling parameters, parallelism, and stop tokens from the recommended settings, and frame your requests with the prompting guide. To fine-tune the model, see Post-training.