Qwen3.8-Flash-Next, DeepSeek's DeepInfer, PyTorch 2.15 on MPS

Two open releases and one tooling update today, all about doing more with the hardware you already have. I read each primary page. I ran none of the code, and the figures are the publishers' own.

Qwen3.8-Flash-Next

Qwen published open weights for what it calls an experimental preview of the architecture that will underpin Qwen4. The model card lists 125B parameters with 6B activated, plus 51B of n-gram embedding and 4B for multi-token prediction. The backbone is 48 layers, laid out as repeating blocks of three Gated DeltaNet layers followed by one Qwen Sparse Attention layer, each paired with a mixture-of-experts layer. There are 512 experts, with 10 routed and 1 shared active per token.

The pitch is efficiency rather than raw size. Qwen Sparse Attention works on micro-blocks instead of individual tokens, which the card says cuts long-context latency, and the n-gram embedding adds parameters in a way that is easier to offload than more experts. Context is 262,144 tokens natively and extensible to 1,000,000. The card names Transformers, vLLM, SGLang and TokenSpeed as compatible, and the license is Qwen Community 1.0, so I would read it before building on it.

DeepInfer

DeepSeek open-sourced DeepInfer under Apache 2.0. The README describes it as the framework behind DeepSeek's large-scale production services, and this release covers offline inference for DeepSeek-V4.1-Flash on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.

The feature list is practical: attention data parallelism, expert parallelism with load balancing, draft-token decoding, asynchronous scheduling on by default, and structured outputs through regex, JSON schema or EBNF grammar. Batch-invariant inference and deterministic sampling are NVIDIA only. The README states what is not included yet: a KV cache system, prefill and decode disaggregation, and load balancing across replicas. It links Pareto charts and a sweep script instead of quoting numbers, so the first useful thing to do is rerun that script.

MPS in PyTorch 2.15

The Hugging Face PyTorch team wrote up the Apple silicon backend in 2.15. FP8 tensors now work, halving storage against BF16, and a native grouped matmul makes mixture-of-experts decoding 5 to 6.5 times faster than 2.14 on an M3 Ultra with torch.compile. Qwen3.6-35B-A3B goes from 9.4 to 61.3 tokens per second, and LFM2.5-8B-A1B from 23.6 to 123.6.

Two smaller results caught my eye. Wan2.2's tiled VAE decode peaks at about 8 GiB of driver memory, down from about 270 GiB. And the same padding change lifts a Qwen-Image-2.1 round trip from 10.06 dB to 29.95 dB PSNR against CPU, which reads as a correctness fix. It needs macOS 15 and Python 3.11 or later.

What I'd watch next

I want independent decode numbers for Qwen3.8-Flash-Next on the same Mac, since 6B active parameters is exactly the shape that suits unified memory. For DeepInfer, I am waiting on the cache and disaggregation pieces, and on someone reproducing the Pareto charts. The PyTorch 2.16 preview promises attention fully in Metal and a smaller install, so that is the next release I will check.