Qwen 3.8-27B on Four 3090s: What a Local Frontier Actually Looks Like

The machine this post was written on

Before the model, the box. Everything in this post — including the words you are reading — was produced by a model that lives on four second-generation consumer cards in the rack next to this desk.

The box is 4× RTX 3090: 96 GB of GDDR6X in total, no ECC, no NVLink, no enterprise support contract. It is the most common form of "big local memory" there is — the 3090 was always $1,500-ish, and four of them have been the de-facto prosumer standard for exactly the reason this post exists: they get you a full-precision 27-billion-parameter model with room to spare.

What runs on it is deliberately boring. The serving stack is our own configuration repo: vLLM 0.23.0 with FlashInfer, tensor-parallelism across all four cards, BF16 weights, FP8 KV cache, 262K context, and Multi-Token Prediction (MTP) speculative decoding with 3 draft tokens. The repo's readme puts the design philosophy in one line: one thing — serve a 27B Qwen in BF16 on 4× RTX 3090.

Why the 27B class just crossed a line

For two years the honest rule was: 96 GB runs a 32B model quantized, or a 70B model heavily quantized. Full precision was the API's privilege. That rule quietly broke. The 27B skeleton is now served at BF16 — the model's native precision, zero quantization — at ~25 GB per card, while leaving the other 90+ GB for a 262K-token KV cache (even after the FP8 compression) and concurrency. In other words: "local" no longer implies a quality tax, at least for the dense frontier of 2026.

The one concession in the whole stack is the KV cache quantization, and even that is a memory decision, not a quality one: long-context agent sessions generate enormous KV tables, and FP8 KV halved their footprint for a precision difference that is below the model's own benchmark noise. If you are deciding between this box and the API, the math section is where the argument gets concrete.

What changed: 3.6 → 3.8

The easy part of the story: nothing. Same skeleton. 64 layers, 27B parameters, hidden dimension 5120, a hybrid of Gated DeltaNet linear attention (48 V / 16 QK heads) interleaved with full Gated Attention (24 Q / 4 KV heads), and the MTP head for speculative decoding. If you are already running 3.6, the weights slot into the same box, the same flags, the same memory footprint.

The delta that matters is in post-training and control surface. Five things, from the card:

• reasoning_effort control. Thinking is on by default — but you can now dial the depth: xhigh, medium, low, or off per request. For an agent fleet this is the single most useful knob in the release: you stop paying xhigh-level tokens on trivial rewrites.
• preserve_thinking, default on. Reasoning blocks from prior turns are retained, which the card points at directly: decision consistency across long agent runs, and better KV-cache reuse because the model stops re-deriving settled context. For a 262K-context box, that is a compounding efficiency win.
• Broader harness compatibility. "More popular harnesses and dev tools" is understatement-grade corporate language for "our VS Code agent and OpenWebUI stopped fighting the formatting." Section two harnesses is the proof.
• Native video understanding. The vision path went from images to hour-scale video, with a documented recipe to lift the frame-sampling ceiling for long clips.
• The score jump. On Qwen's own table, DeepSWE 1.1 goes 13.3 → 42.2. That is not a version bump, that is a different product wearing the same weights-shape.

Running it: BF16 on four 3090s

The numbers from the box, not from the press release. All from the configuration repo, all measured on Ubuntu 24.04, vLLM 0.23.0 + FlashInfer, TP=4.

Three of those rows deserve attention. First-token latency of ~5 ms on a warm session is faster than the human blink between two keystrokes — the model feels like a very fast autocomplete, which matters more than headline tok/s for an agent loop that hammers the API hundreds of times a task. Throughput is flat across 100→1,000 token outputs (74.6 → 74.0 → 74.7), meaning the MTP drafter is doing its job and long generations do not degrade. And per-GPU occupancy stays under 25 GB: with VLLM_GPU_MEM 0.90 and VLLM_MAX_SEQS 2, the four cards sit comfortably under cap even with a full 262K-cache session live.

The quantization callout, in the open

Prefix caching is the other quiet hero. Agent sessions are heavily re-prefixed — system prompt, repo context, prior turns — and with caching on, the repeated prefix costs ~5 ms instead of seconds. This is why the agent work in Sections 6 and 7 feels the way it feels.

The local class of August 2026

"Best model that fits in 96 GB" is a claim about a class, so here is the class — the other models a single prosumer box can hold, and where each one wins.

Muse Glimmer-30B: the one to beat

Meta Superintelligence's Muse Glimmer-30B (August 2026) is the release that makes this post honest rather than promotional — Qwen's own 3.8 card puts it in the comparison table as the main event. It is a dense ~29.6B with a 1.8B ViT-G/14 vision encoder, 52 layers, 131K+ context, Apache 2.0, and it is designed around consumer cards: the headline is that its 4-bit build fits 24 GB with the quantization cost explicitly measured (0.2–1.0%), and a DFlash 16-token speculative drafter takes it from 74.9 to 233.4 tok/s on an RTX 5090 — a 3.1× speedup out of the box.

On scores, it is genuinely competitive in parts of the table — SWE-bench Verified 76.0 is the best local number in the class — while sitting well behind on long-horizon agentic work: Terminal Bench 2.1 at 51.7 versus Qwen3.8's 73.0, and HLE at 22.0 versus 30.8. One-liners for the rest of the class: GLM 5 punches far above its weight on the official SWE-bench Verified leaderboard (72.8, high-effort) but does not report the full agentic table; Gemma 4 31B is Google's single-card 31B, strong on reasoning benchmarks, no public agentic-coding numbers for the 31B cut; GPT-OSS-120B reminds you why 120B still wins raw code on the leaderboard (26.0) while remaining the wrong size for a 96 GB box.

The software-benchmark gap

So where does a model that lives in a rack actually stand against the closed frontier? This is the chart the whole post builds toward: agentic coding, the benchmark family that separates a model from an agent.

Read the chart left to right and the story writes itself. On SWE-bench Pro — the hardest agentic-coding benchmark in the family, real-world multi-file SWE tasks from major repos — Qwen3.8-27B at 61.7 is not just the best local model in the table, it is ahead of the closest frontier anchor shown (Opus 4.6 Max at 53.4) by 8.3 points. On LiveCodeBench v6, at 90.3, it leads the whole column including the frontier. On Terminal Bench 2.1 it is second to Opus 4.6 Max (78.2 vs 73.0) and ahead of the prior generation by nearly a full 10 points.

Now the honest half of the picture, which we would be doing you a disservice to skip:

What the chart is saying, net-net: the 27B class in August 2026 is within frontier range on agentic coding, occasionally ahead on specific rows, and the remaining gap is shrinking at the pace of one model release per quarter on both sides. That is a different sentence than it was twelve months ago.

Two harnesses, one platformer

Benchmarks are what a model says it can do. This section is what it actually did, on the tasks practitioners actually run — the same one-line prompt, through two different agent harnesses, recorded.

The prompt, verbatim, in the VS Code agent: Create a small 2d platformer in a single html. The model chip in the harness reads qwen3.8-27b FP16 · Extra High. That is the whole configuration. What follows is the agent thinking, planning, and writing a complete, working, self-contained platformer — one HTML file, canvas rendering, physics, a win state — then running it and iterating until it plays.

Same session, later: the game plays. "Pixel Peak," a tiny platformer, runs to its flag, and the win screen shows up on its own.

Now the A/B. The same task re-run in OpenWebUI — a different harness, a different rendering context, no IDE. The model ships "Ember Peak," a fully styled variant (torch-lighting, summit finish) that plays clean to a SUNSET CROWNED win in 0:15.

The point is not "the model made a game." The point is: on the task class a practitioner actually runs daily — "write me a small tool / page / thing, end to end, in one file" — the local model on this box is operating at the level the frontier API does, and the harness choice now matters more than the model choice.

The hybrid locomotive simulators

The platformer is the demo. The locomotive is the artifact. This is what the box can now build: not a game, but an engineering instrument.

Over a sustained working session — plan, architecture, physics, UI, iteration, across many agentic turns — the model produced a self-contained web page simulating a HE-7531 hybrid-electric locomotive: a 6.4 MW continuous diesel prime mover, a 2.1 MWh battery pack, 94% regenerative-braking energy capture, and 3.2 MW of dynamic-braking resistor dissipation, all running live in the browser with no backend. Alongside it, a UNIT 4471 classic diesel-electric variant (16-cylinder prime mover, no battery) for side-by-side comparison. The dashboard mode — "LIVE SYSTEM CUTAWAY" — is an animated system diagram with live gauges: torque, battery state-of-charge, regen-current, resistor dissipation, all updating in real time from the simulation state.

What this artifact proves, in the language of this post: a 27B model at native precision, on 96 GB of second-hand consumer VRAM, is capable of end-to-end generation of a complex, multi-component, physically-grounded interactive system — planning, writing, testing, iterating, shipping — without ever leaving the box. The SWE-bench Pro 61.7 in Section five is not an abstraction; the HE-7531 is the receipt.

What we run it on

The proof of a local model is not a demo. It's what it does when it's in production, on the work that's actually on the plate.

This site's research pipeline is the first production load. The create-research-post skill — the two-gate human-in-the-loop workflow that produced this very post, including the research, the code, the media re-encoding, the CDN uploads, and the validation suite behind each gate — is driven by the local 3.8 as its primary reasoning engine. The model writes the post, the agent runs the build, the human approves the gates. That loop is the "agent-first" part of the business model, and it's running on the rack, not on an API.

The second production load is the agent-first marketing agency this site is the research arm of: client site work, research, deployment, and iteration, all driven by local 3.8 agents through the same harnesses shown in Section two harnesses. The concrete examples: the VS Code agent building and shipping single-file web tooling; the OpenWebUI harness driving multi-turn research and content sessions; the MTP-served model keeping first-token latency under 5 ms so that the agent loop — which makes hundreds of tool calls per task — feels like it's talking to a co-worker, not an API.

The math of staying local

The objection is always the same, and it deserves a direct answer: why not just call the API?

The honest cost picture for a 4× 3090 box at our actual agent volume:

The API alternative, at comparable quality and comparable volume, scales linearly with every token the agent fleet generates — and agent workloads are not "chat" volume, they are hundreds of prefill-heavy tokens per tool call, multiplied across parallel sessions, multiplied across the workday. At the scale of a research pipeline plus an agency's daily agent traffic, the API bill is not a rounding error; it is a line item that compounds with the business. The box stops compounding after purchase. The ~2.5 kW draw is the real ongoing cost, and at our electricity rate it is a fraction of what the API line item would be at the same volume.

Two properties of local that no API price can match, and that are not about money:

• Data never leaves. Client work, research corpora, internal workflows — none of it crosses a network boundary to a third party's inference fleet. For an agency handling client data, that is a contractual and liability property, not a preference.
• Latency is a physical property, not a pricing tier. First token at 5 ms means the agent loop's thinking time is the model's thinking time, not a network round trip. That is why the harnesses in Section two harnesses feel the way they do — the round trip has been eliminated, not optimized.

The honest counterweight, before anyone files an objection: the frontier still wins on the hardest single-shot reasoning tasks (Section five's warning). For those tasks, the API is still the tool. The local box is the tool for the volume workload — the 95% of agent traffic that is tool-calling, file-editing, and long-context reasoning where the quality gap has closed to inside the harness noise. That split is the real answer to "why not just use the API": you do, for the 5%.

Bottom line

Qwen3.8-27B is, at the moment, the strongest general-purpose model that fits in roughly 96 GB of consumer GPU memory — and the claim is earned, not asserted.

The evidence map, mapped to the sections:

• The delta (Section two): same skeleton, but reasoning_effort, preserve_thinking, harness compatibility, and video input, with DeepSWE 1.1 jumping 13.3 → 42.2. A version bump would not do that.
• The class (Section four): against Muse Glimmer-30B, GLM 5, and Gemma 4 31B, Qwen3.8-27B wins on agentic-coding rows and context; Glimmer wins on footprint. The class is real, and it is competitive.
• The gap (Section five): on the official SWE-bench Verified leaderboard the frontier keeps a 3–6 point lead; on SWE-bench Pro the local model is ahead of the closest frontier anchor shown in the same table. The gap is real and it is shrinking.
• The footage (Sections 6–7): two harnesses, one prompt, two playable games; and a full engineering simulator built end-to-end, in production, on the box.
• The math (Section nine): volume workload at zero per-token cost, data that never leaves, latency as a physical property — with the honest 5% that still deserves the API.

Key Sources and References

• Qwen3.8-27B model card — architecture, features, and the 3.8/3.6/Qwen3.7-Plus/Muse Glimmer-30B/Opus 4.6 Max benchmark table: huggingface.co/Qwen/Qwen3.8-27B
• Qwen3.6-27B model card — baseline architecture and SWE-bench Verified 77.2, Terminal Bench 2.0 59.3, context 262,144: huggingface.co/Qwen/Qwen3.6-27B
• Muse Glimmer-30B model card (Meta Superintelligence Lab, Aug 2026) — class peer, 4-bit 24 GB, DFlash 16-token drafter, 0.2–1.0% 4-bit degradation: huggingface.co/meta-models/Muse-Glimmer-30B
• Official SWE-bench Verified leaderboard (bash-only scaffold) — frontier tier numbers: swebench.com
• Qwen 3.8 team blog — "Qwen3.8-Max: A New Bar for Coding and Cowork": qwen.ai/blog
• Our serving configuration, vLLM 0.23.0 + FlashInfer, TP=4, BF16 + FP8 KV, MTP 3, measured 74.6 tok/s and ~5 ms TTFT: github.com/michellacle/4x_RTX3090_qwen3.8-27b
• The hybrid locomotive simulators, HE-7531 + UNIT 4471, generated by the local 3.8, live: michellacle.github.io/hybrid-locomotive
• First-hand footage — harness-comparison and locomotive-animation videos (the underlying MP4s are embedded above; these links are for context): X: harness comparison · X: locomotive animation