Apple released MLX v0.30.1 with distributed inference support. It lets users run Muse Spark Open on a cluster of Mac Studios. The model reaches 36 tokens per second on four machines. A single M5 Ultra Mac Studio speeds up time to first token by up to 4.6× over M3 Ultra. It also improves decode speed by 1.6×. The M5 Ultra uses 300W peak power during inference. Pipeline parallelization splits models across depth. Tensor parallelization splits models across width. Both rely on RDMA over Thunderbolt 5. MLX provides `mlx.launch` to run jobs across multiple nodes. Muse Spark Open fits only when quantized to 4-bit. Two-cable Thunderbolt links cut TTFT for large contexts. The update targets researchers and developers.
What changed
Large models like Muse Spark Open required more than 1.6TB RAM and could not run on a single Mac Studio.
What it unlocks
Running Muse Spark Open and other frontier models on a cluster of Mac Studios via MLX distributed inference.
- Muse Spark Open: 816B parameters, 42B active
- Muse Spark Open: 36 tokens/sec on 4 Mac Studios
- M5 Ultra: 1.2 TB/s memory bandwidth
- M5 Ultra TTFT: up to 4.6× faster vs M3 Ultra
What you need to act on it
- Mac Studio with M5 Ultra
- macOS 26.2 or later
- MLX v0.30.1 or later
- Thunderbolt 5 RDMA driver
- Two or more Mac Studios for distributed inference
Sources