The headline is real
Apple published a new MLX cluster study on October 15. Four M5 Ultra Mac Studios ran the 816 billion parameter Muse Spark Open model at 36.73 tokens per second with an 8,192 token context. The same setup reached 34.66 tokens per second with 65,536 tokens of context. Apple says each result generated 128 tokens after the prompt.
The cluster uses Thunderbolt 5 with RDMA and MLX's JACCL communication backend. Apple's example benchmark and server commands set MLX_METAL_FAST_SYNCH=1. The article calls the feature experimental and says it reduces CPU to GPU synchronization overhead.
The speed number is evidence for the measured route. It is not a reliability grade for every switch in that route.
The warning lives one link away
The MLX distributed documentation gives the flag a sharper label. It says fast sync is unreliable, can deadlock, and can leave the GPU wedged. The docs say the flag is off by default and best left unset. That warning links to MLX issue 3142.
The issue remains open and carries a wontfix label. An MLX maintainer wrote that Metal does not support atomic memory coherence between the CPU and GPU for this use, and that no public method can implement a GPU spinlock guaranteed to work across all GPUs. Reports in the thread describe locks, stalls, and restarts under specific distributed workloads. Those reports do not establish a failure rate for Apple's benchmark.
A benchmark can accept a different risk
A bounded benchmark has a start, a short measurement window, and an operator nearby. A coding-agent server may run for hours, carry live repository state, and receive requests without someone watching the console. A rare wedge has a different cost there.
Apple includes fast sync in both a benchmark example and an mlx_lm.server example. That is a useful demonstration. It is not a production policy. Before copying the server command, decide who detects a hang, who can stop every rank, what state survives, and how long a full cluster restart takes.
Keep the default boring
Start with the flag unset. Record the MLX revision, macOS build, model revision, parallel layout, cable map, prompt shape, and generation length. Run the same workload with the flag unset and enabled. Compare time to first token, decode rate, output checks, stalls, process exits, and recovery time.
Put the deadline outside the MLX processes. A wedged process cannot be trusted to enforce its own timeout. Test repeated long-context requests and concurrent work because issue 3142 describes failures that did not appear in every run. Keep a restart procedure that covers every rank and the client that called them.
Promote evidence, not a switch
- Keep fast sync unset in the service baseline.
- Pin one matched benchmark packet before enabling it in a test.
- Use an external deadline and a stop path that reaches every host.
- Repeat the stress run. One clean pass only proves one clean pass.
- Record output checks and recovery time beside speed.
- Move the flag into a service only after the owner accepts the failure cost.