Hardware platforms

Run Muse Spark Open on NVIDIA, AMD, Intel, and Apple hardware. Each section lists the checkpoints, parallelism, and runtime for its platform, with sample commands. Pick a platform to jump to it.

Vendor	Accelerator	Checkpoints	Runtime
NVIDIA	Blackwell, such as B300 and GB300 (4 GPUs for NVFP4, 8 for BF16)	BF16, NVFP4	SGLang, vLLM, TensorRT-LLM, NVIDIA Dynamo
AMD	Instinct MI355X or MI350X (8 GPUs)	BF16, MXFP4, FP8	vLLM
Intel	Xeon 6 with P-cores, such as the 6980P	INT4	SGLang
Apple	Mac Studio with M5 Ultra (2 × 512 GB or 4 × 256 GB of unified memory)	NVFP4	MLX

For SGLang and vLLM on NVIDIA GPUs, see Run inference.

NVIDIA

Validated serving configurations for Muse Spark Open are maintained in the runtime recipe databases; start from those rather than hand-tuning.

TensorRT-LLM: the pareto-optimized serving configuration database at examples/configs/database/(opens in new tab), the Muse Spark Open deployment guide(opens in new tab), and the speculative-decoding deployment guide(opens in new tab). See also Deploy with TensorRT-LLM.
Dynamo deployment recipes: the Muse Spark Open Dynamo recipe(opens in new tab). See also Deploy with NVIDIA Dynamo.

Hardware notes not covered by the recipe sources:

Checkpoint	Accelerator	GPUs	Parallelism	KV cache	Runtime
meta-models/Muse-Spark-Open-816B-A42B-NVFP4	GB300 NVL (`SM103`)	4	TP4, EP4	FP8	TensorRT-LLM

The NVFP4 checkpoint fits on a single NVIDIA Blackwell (SM100 family) 4-GPU node, with attention tensor-parallel 4-way and the 256 routed experts sharded 4-way. The configuration above was validated on GB300 NVL with TensorRT-LLM. The FP8 KV cache is required on SM103 when chunked prefill or KV-cache block reuse is enabled, and the recommended TensorRT-LLM configuration enables both; see Deploy with TensorRT-LLM.

AMD

Checkpoint	Accelerator	GPUs	Parallelism	KV cache	Runtime
meta-models/Muse-Spark-Open-816B-A42B	MI355X or MI350X	8	TP8	GPU	vLLM
amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4	MI355X or MI350X	8	TP8+EP8	GPU	vLLM
amd/Muse-Spark-Open-816B-A42B-Quark-PTPC-FP8	MI355X or MI350X	8	TP8+EP8	GPU	vLLM

Sample commands

BF16

bash
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_MUSE_MOE_REDUCE_DTYPE=bf16
COMMON_ARGS="--host 127.0.0.1 --port 8006 --served-model-name musespark \
  --tokenizer-mode hf --config-format auto --load-format safetensors --generation-config vllm \
  --language-model-only --dtype bfloat16 \
  --tensor-parallel-size 8 --distributed-executor-backend mp --attention-backend ROCM_AITER_FA \
  --max-model-len 32768 --max-num-batched-tokens 4096 --gpu-memory-utilization 0.95 \
  --enable-chunked-prefill \
  --reasoning-parser muse --tool-call-parser muse --enable-auto-tool-choice"
CG='--compilation-config {"cudagraph_mode":"FULL_AND_PIECEWISE"}'
vllm serve meta-models/Muse-Spark-Open-816B-A42B \
  $COMMON_ARGS $CG --moe-backend triton --max-num-seqs 32

BF16 checkpoint with online MXFP4 experts

bash
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_MUSE_MOE_REDUCE_DTYPE=bf16
COMMON_ARGS="--host 127.0.0.1 --port 8006 --served-model-name musespark \
  --tokenizer-mode hf --config-format auto --load-format safetensors --generation-config vllm \
  --language-model-only --dtype bfloat16 \
  --tensor-parallel-size 8 --distributed-executor-backend mp --attention-backend ROCM_AITER_FA \
  --max-model-len 32768 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --reasoning-parser muse --tool-call-parser muse --enable-auto-tool-choice"
CG='--compilation-config {"cudagraph_mode":"FULL_AND_PIECEWISE"}'
VLLM_ROCM_MUSE_MOE_ONLINE_QUANT=mxfp4 vllm serve meta-models/Muse-Spark-Open-816B-A42B \
  $COMMON_ARGS $CG --max-num-seqs 128

MXFP4

bash
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_MUSE_MOE_REDUCE_DTYPE=bf16
COMMON_ARGS="--host 127.0.0.1 --port 8006 --served-model-name musespark \
  --tokenizer-mode hf --config-format auto --load-format safetensors --generation-config vllm \
  --language-model-only --dtype bfloat16 \
  --tensor-parallel-size 8 --distributed-executor-backend mp --attention-backend ROCM_AITER_FA \
  --max-model-len 32768 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --reasoning-parser muse --tool-call-parser muse --enable-auto-tool-choice"
CG='--compilation-config {"cudagraph_mode":"FULL_AND_PIECEWISE"}'
vllm serve amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 \
  $COMMON_ARGS $CG --max-num-seqs 128

Throughput mode:

bash
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_MUSE_MOE_REDUCE_DTYPE=bf16
COMMON_ARGS="--host 127.0.0.1 --port 8006 --served-model-name musespark \
  --tokenizer-mode hf --config-format auto --load-format safetensors --generation-config vllm \
  --language-model-only --dtype bfloat16 \
  --tensor-parallel-size 8 --distributed-executor-backend mp --attention-backend ROCM_AITER_FA \
  --max-model-len 32768 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --reasoning-parser muse --tool-call-parser muse --enable-auto-tool-choice"
CG='--compilation-config {"cudagraph_mode":"FULL_AND_PIECEWISE"}'
vllm serve amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 \
  $COMMON_ARGS $CG --max-num-seqs 128 --enable-expert-parallel

Lowest latency mode:

bash
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_MUSE_MOE_REDUCE_DTYPE=bf16
COMMON_ARGS="--host 127.0.0.1 --port 8006 --served-model-name musespark \
  --tokenizer-mode hf --config-format auto --load-format safetensors --generation-config vllm \
  --language-model-only --dtype bfloat16 \
  --tensor-parallel-size 8 --distributed-executor-backend mp --attention-backend ROCM_AITER_FA \
  --max-model-len 32768 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --reasoning-parser muse --tool-call-parser muse --enable-auto-tool-choice"
CG='--compilation-config {"cudagraph_mode":"FULL_AND_PIECEWISE"}'
VLLM_KV_CACHE_LAYOUT=LBHNC vllm serve amd/Muse-Spark-Open-816B-A42B-Quark-MXFP4 \
  $COMMON_ARGS $CG --max-num-seqs 32 \
  --speculative-config '{"method":"dspark","model":"Inferact/Muse-Spark-Open-DSpark","num_speculative_tokens":7}'

FP8

bash
export VLLM_ROCM_USE_AITER=1
export SAFETENSORS_FAST_GPU=1
COMMON_ARGS="--host 127.0.0.1 --port 8006 --served-model-name musespark \
  --tokenizer-mode hf --config-format auto --load-format safetensors --generation-config vllm \
  --language-model-only --dtype bfloat16 \
  --tensor-parallel-size 8 --distributed-executor-backend mp --attention-backend ROCM_AITER_FA \
  --max-model-len 32768 --max-num-batched-tokens 8192 --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --reasoning-parser muse --tool-call-parser muse --enable-auto-tool-choice"
CG='--compilation-config {"cudagraph_mode":"FULL_AND_PIECEWISE"}'
vllm serve amd/Muse-Spark-Open-816B-A42B-Quark-PTPC-FP8 \
  $COMMON_ARGS $CG \
  --max-num-seqs 128 \
  --enable-expert-parallel \
  --moe-backend triton

Intel

Checkpoint	Hugging Face repo	Hardware	Runtime
INT4	`intel-ai/Muse-Spark-Open-816B-A42B-INT4`	Intel Xeon 6 with P-cores, such as the 6980P	SGLang

bash
python -m sglang.launch_server \
  --model-path intel-ai/Muse-Spark-Open-816B-A42B-INT4 \
  --quantization gptq \
  --tp-size 2 \
  --reasoning-parser muse --tool-call-parser muse

Apple

Checkpoint	Hardware	Runtime
meta-models/Muse-Spark-Open-816B-A42B-NVFP4	Two Mac Studio computers with M5 Ultra and 512 GB of unified memory each, or four with 256 GB each	MLX

MLX splits the model across the Macs over RDMA on Thunderbolt 5, which needs macOS 26.2 or later and a direct link between every pair of Macs. For setup and commands, see Run inference › MLX.

Next steps

For every runtime and its full flag set, see Run inference. To size parallelism and context length for your workload, see Recommended settings.