GPT-OSS-120B is 59 GiB on disk. Godwit runs it on an Apple Silicon Mac with 16 GB of memory by keeping only the shared trunk resident and streaming mixture-of-experts weights from SSD as the router asks for them. Swift and Metal, no dependencies. It also runs GPT-OSS-20B and Qwen3-30B-A3B from the same binary, switchable without restarting.
GPT-OSS-120B has 128 experts in each of its 36 layers. For any given token the router picks four of them. The other 124 contribute nothing to that token and need not be in memory.
Every mainstream runtime still requires the whole checkpoint resident, so disk footprint sets the memory requirement and a 59 GiB model needs a 64 GiB machine. Godwit keeps the trunk — embeddings, attention, routers, norms — in 2.12 GiB, and fetches experts as they are chosen.
Embeddings, attention, routers and norms, quantised to int8.
Eight slots per layer, evicted least-frequently-used.
4,608 experts at 12.6 MiB each, read on demand.
The cost is throughput. You are trading tokens per second for the ability to run the model at all, and on a base MacBook Air that buys about 1.4 tokens per second. The bet is that slow beats impossible.
godwit serve opens a dashboard on loopback. No npm, no build step —
the binary serves the whole interface.
A range map is the ornithologist's chart of where a species is found. This is the same idea over topic space. Godwit probes the router with twelve kinds of text and records which experts fire for each; position comes from the principal components of those affinity vectors. Nothing here is trained — it is measurement.
loading…
The counts are of experts that fired often enough
to mean it. An expert selected twice, both times on Python, scores as a
pure specialist on two samples — noise in the costume of a finding. A label is
credited only above 24 activations, twice the number of topics, below which an
expert averages under two observations per topic. Under-sampled experts are
still drawn, faintly, because they are real routing; they just are not counted.
On this model that bar excludes 1,775 of the 4,475 plotted experts, and takes
the python count from 516 to 248. An earlier version of this page
reported the larger number.
| GPT-OSS-120B | GPT-OSS-20B | Qwen3-30B-A3B | |
|---|---|---|---|
| Layers × experts | 36 × 128 | 24 × 32 | 48 × 128 |
| Active per token | 4 | 4 | 8 |
| Install | 59.2 GiB | 11.2 GiB | 16.1 GiB |
| Resident | 5.7 GiB | 4.1 GiB | 2.5 GiB |
| Decode | 1.4 tok/s | 2.8–3.2 tok/s | 2.3 tok/s |
GPT-OSS-20B ran first time with nothing but a spec written down, which proved less than it looked: it shares tensor names, RoPE, activation, sinks and tokeniser with the 120B. Qwen3 shares almost none of that, and it is the reason the claim below changed.
Two of Qwen3's differences would have run
without complaining. A model with no attention sinks cannot be given
a sink of zero — the sink enters the softmax denominator as
exp(sink − max), so zero adds a whole unit of
mass to every row rather than nothing. And MXFP4 has to be written rather than
copied, because Qwen3 ships BF16 where GPT-OSS ships weights already on the
4-bit grid. godwit check-mxfp4 re-encodes OpenAI's own bytes and
demands equality across 2,073,600 blocks; it caught a floored exponent and a
dropped sign bit.
| Metric | Value |
|---|---|
| Decode | 1.4–1.5 tok/s |
| Prefill | ~2.2 tok/s |
| Time to first token | 10–13 s |
| Resident memory | 5.7 GiB |
| GPU busy | 17.6% of wall time |
| Expert reads | 71.6% of decode |
That is device speed, not an inefficiency. The read path already runs at what this SSD delivers. Small Apple SSDs use fewer NAND dies and are genuinely slower — this one reads ~2 GiB/s, where 512 GB–2 TB modules reach 3–6. The same code should reach roughly 3–5 tok/s on a Mac with a larger SSD. That has not been verified, because no such machine was available.
Recorded because they cost as much to establish as the wins.
| Attempt | Result |
|---|---|
| Prefetch by reusing previous layer's expert IDs | 4.8% hit vs 3.1% chance |
| Concurrent miss reads | no effect |
| Splitting reads into chunks | slightly worse |
| More cache slots (24) | swaps; 0.12 tok/s |
| Threadgroup-memory staging | no effect |
| File defragmentation | no effect |
It is slow. 1.4 tokens per second is roughly 100 words a minute — slower than you read. Good for long-form generation you can walk away from; poor for conversation.
GPT-OSS-20B fits in the same 16 GB and runs at 30–50 tok/s under MLX. For most tasks it is good enough, and if that is true for yours you should use it instead. Godwit matters where the larger model's quality is worth the wait, or where even 20B will not fit.
This page used to predict that adding a second family would mean writing a loader rather than a runtime. That was half right. Tensor naming did become data, and the spec really does drive dimensions — but Qwen3 also needed new kernels: QK-norm, plain SwiGLU, an attention path compiled without sinks, and an MXFP4 encoder, because it ships BF16 rather than weights already on the 4-bit grid. Seven changes where five were predicted, and two of the seven were not on the list at all. A third family will probably find something too.
# Apple Silicon, macOS 26+, Swift 6.2+
# Free space: ~60 GB for the 120B, ~17 GB for Qwen3, ~12 GB for the 20B
git clone https://github.com/rayl15/Godwit.git
cd Godwit
swift build -c release
# Stream and repack the checkpoint. ~59 GiB, about 3 hours.
.build/release/godwit install --output model.gwt
# Or a smaller one — 16 GiB, 105 minutes. The dashboard offers
# whichever installs it finds beside the one you point it at.
.build/release/godwit install --output qwen3.gwt --model-id Qwen/Qwen3-30B-A3B
.build/release/godwit chat --model model.gwt # terminal
.build/release/godwit serve --model model.gwt # dashboard