# GVS5H self-orchestration, Cerebras at trillion scale, MemT-1.0

> A training-free coding orchestration repo, how Cerebras serves a 3T-parameter model across wafers, and a memory-native model paper.

Published 25 Oct 2026. Coverage: 24 Oct 2026 to 25 Oct 2026. By Ali Asif.

## Items

1. **GVS5H: ledger-based zero-shot self-orchestration** (slee-persis, Agents and coding): MIT-licensed repo where fresh instances of one model split a coding problem through shared files, reporting 93.0% pass@1 on 100 hard LiveCodeBench problems with Qwen3.8 Flash Next. [Primary source](https://github.com/slee-persis/GVS5H)
2. **How we deploy trillion-parameter frontier models on Cerebras** (Cerebras, Frontier): Worked example of serving Kimi K3 across 59 WSE-3T wafers with one layer group per wafer, claiming 1,000 to 2,000+ tokens per second. [Primary source](https://www.cerebras.ai/blog/how-we-deploy-trillion-parameter-frontier-models-on-cerebras)
3. **MemT-1.0: Towards Memory-Native Evolution** (MemTensor, Science): Paper on building memory into the model with Gated DeltaNet-2, reporting about 1.86x inference speedup and 66x less persistent storage than a full-context KV cache at 128K tokens. [Primary source](https://arxiv.org/abs/2610.26825)

Not a day of big model launches, but three primary pages are worth the read: a repo on orchestration, a serving write-up, and a paper on memory. I opened each one. I did not run any of them.

## GVS5H, one model coordinating copies of itself

The repo tests what it calls ledger-based zero-shot self-orchestration. There is no training. Fresh instances of a single model split up a coding problem and coordinate through a shared filesystem, where files hold the plan, the notes and the current solution.

The headline numbers come from the 100 latest hard LiveCodeBench problems across eleven open and closed-weight models. Orchestrated Qwen3.8 Flash Next reaches 93.0% pass@1, against 90.4% for Claude Fable 5 as the baseline, at 9% of the cost. Locally served Qwen3.8-27B rises from 66.8% to 92.4%. The page also says the gains are not universal, and some models are unchanged or worse. Its transcript analysis credits decomposition and persistent context.

Code is MIT, and the paper, figures and run data are CC BY 4.0. You run it with uv and an API key, with one engine flag for the multi-agent path and one for the single-call baseline. These are the author's own results on one benchmark, so I would rerun on my own tasks before believing the cost claim.

## Cerebras on serving a 3T model

Cerebras walks through serving Kimi K3, an open model of about 3 trillion parameters, across multiple WSE-3T wafers. Each wafer has 44 GB of on-chip memory. With 4-bit weights, an 8-bit KV cache and 256k average requests, the weights split across 93 layers of about 16 GB each.

Each full-attention layer gets its own wafer, with the leftover memory used for its KV cache. Linear-attention layers need no per-token cache, so two share a wafer. Kimi K3 has 69 linear and 24 dense layers, which gives 35 plus 24, or 59 wafers. Only activations move between wafers, over RDMA on Ethernet at roughly 2 microseconds per hop, and each wafer works as a pipeline stage on a different request.

The page claims 1,000 to 2,000+ tokens per second and argues that tokens per second per megawatt, compared at equal output speed, is the metric that matters. The concurrency figure sits in an image, so I cannot quote it. The Kimi K3 benchmark scores on the page are Moonshot's, not Cerebras measurements.

## MemT-1.0, memory inside the model

The paper builds memory into the model computation using Gated DeltaNet-2 with a token-level update, and adds Mixture-of-Storage to scale capacity. Training uses 34B tokens in two stages. Two models are named, MemT-1.0-Nano and MemT-1.0-Flash.

At a 128K-token history, Nano reaches about 1.86x end-to-end inference speedup and a 66x reduction in persistent storage versus a full-context KV cache. The abstract says the models beat parametric memory baselines on average and stay competitive with retrieval-augmented methods. The arXiv page links no code or weights.

## What I'd watch next

Whether anyone reproduces the GVS5H cost comparison on a different benchmark, and which models are the ones that get worse. Whether Cerebras publishes the concurrency numbers in text. Whether MemT-1.0 ships weights so the 66x storage claim can be checked.

## My take

GVS5H is the one I would try first, because it is MIT code and a run costs an API key and an afternoon. The Cerebras post is an architecture explainer more than a launch, and the MemT paper links no weights yet. I read the three pages and ran none of it, so every number belongs to its authors.

## Sources

- https://github.com/slee-persis/GVS5H
- https://www.cerebras.ai/blog/how-we-deploy-trillion-parameter-frontier-models-on-cerebras
- https://arxiv.org/abs/2610.26825

Canonical: https://heyaliasif.com/ai/2026-10-25-gvs5h-cerebras-trillion-memt