Analysis Local AI
Local models for coding vs Claude: how they actually perform and what they really cost
Qwen3.6, Devstral, gpt-oss and Muse vs Claude for coding: SWE-bench results, the hardware you need, and the math in euros for local versus the API.
Tested on October 17, 2026 in llama.cpp b11714 (CPU x64) · Windows Server 2022, Ryzen 9 5900X with 3 vCPUs, 12 GB, no GPU. We only tested small models (0.8B to 4B) on our VPS. Coding models of 20B or more don't fit on it: their results are the ones published by each vendor, and the speeds come from the measurements we cite. How we test
In this article
The best coding model you can run at home on a single 24 GB GPU, Qwen3.6-27B, solves 77.2% of SWE-bench Verified according to Qwen. That’s nearly what Claude Opus 4.5 scored (80.9%), but Opus 4.5 is no longer Anthropic’s latest, and no Claude model can run locally. For autocomplete, scripts, and code that can’t leave your machine, a local model is worth it. For agents that work on their own for an hour and touch lots of files, Claude is still in another league. Money-wise, the math only favors local if you already own the hardware or you code many hours a day.
This is what we found by cross-checking the data published as of October 17, 2026, with all the math done in euros. What we could run ourselves were small models, and that result is part of the answer too.
The coding models that actually fit on one machine
We left out the large open models that come close to the frontier, like DeepSeek V4-Flash (284 billion parameters), GLM-5.2 (753 billion), or Kimi K3: they’re open, but they need a server with several GPUs. These are the ones that fit on a PC with one GPU or on a Mac:
| Model | Type | SWE-bench Verified | Approximate memory (4-bit) | License | Reasonable minimum hardware |
|---|---|---|---|---|---|
| Qwen3.6-27B | Dense | 77.2% | ~18 GB | Apache 2.0 | 24 GB GPU or Mac with 32 GB |
| Muse Glimmer-30B (Meta) | Dense | 76.0% | ~17 GB | Apache 2.0 | 24 GB GPU or Mac with 32 GB |
| Qwen3.6-35B-A3B | MoE, 3B active | 73.4% | More than the 27B | Apache 2.0 | 24 GB GPU, or a small GPU with 32 GB of RAM |
| Devstral Small 2 (Mistral) | Dense, 24B | 68.0% | Somewhat less than a 27B | Apache 2.0 | 16-24 GB GPU |
| gpt-oss-120b (OpenAI) | MoE, 5.1B active | 62.4% (high reasoning) | 64 GB with 8,000 tokens of context | Apache 2.0 | Mac or mini PC with 96-128 GB, or 12 GB GPU with 64 GB of RAM |
| gpt-oss-20b (OpenAI) | MoE, 3.6B active | 60.7% (high reasoning) | 14.9 GB with 8,000 tokens of context | Apache 2.0 | 16 GB GPU, or 12 GB with some in RAM |
| Reference: Claude Opus 4.5 | — | 80.9% | Doesn’t run locally | Commercial | — |
SWE-bench figures as stated by each vendor in its model card or announcement; the Claude Opus 4.5 figure appears in the comparison table on the Qwen3.6-27B model card. The gpt-oss memory figures are the ones measured by llama.cpp with everything loaded.
Three caveats before drawing conclusions:
- Anthropic no longer publishes SWE-bench Verified for its current models (Opus 5.5 and Sonnet 5.5). It uses SWE-bench Pro, which is harder. That’s why we compare against Opus 4.5, the last one with a Verified score: the reference point is an Anthropic model from a year ago, not today’s.
- Each vendor measures with its own setup (the “harness”: the tools, the number of attempts, the step limit). Two figures from different vendors look like a comparison, but they aren’t quite one.
- Qwen’s newest model, Qwen3.8-27B (August 2026, Apache 2.0), doesn’t report a Verified score, only SWE-bench Pro: 61.7. It looks better than 3.6, but we can’t put it in the same column.
What it means day to day
SWE-bench Verified measures whether the model can fix a real bug in a Python repository: it reads the issue, finds the files, changes the code, and passes the tests. 77% is a lot. But the experience of people who have used these models daily, like the Failing Fast analysis we cite in the sources, agrees on how the work splits:
Where a local 24-30B model does well:
- Autocomplete in the editor, where latency matters more than intelligence.
- Self-contained functions and scripts: “convert this CSV,” “add a test to this function.”
- Explaining code and writing documentation.
- Code under a nondisclosure agreement or data that can’t leave your network.
Where the gap with Claude shows:
- Changes that touch many files at once and require understanding the architecture.
- Long agent sessions: dozens of tool calls in a row without losing the thread.
- Subtle bugs (concurrency, shared state, edge cases).
- Truly long context. Local models advertise 128,000 or 262,000 tokens, but every token of context takes up memory in the KV cache. With a 24 GB GPU and an 18 GB model, the room left for context runs out sooner than you’d expect.
And below that size, the drop is steep. We checked it on our GPU-less VPS: we asked Qwen3.5 2B and Qwen3.5 0.8B for a Python function that totals file sizes by extension. The 2B forgot the import os and overwrote instead of adding, both times we tried. The 0.8B returned a function that didn’t even take a list. The 4B came very close (it imported os and added correctly), but it unpacked three values from a function that returns two, so it failed on the first file that actually existed. Only Gemma 4 E2B got it right on the first try: one of the five models we asked. The details are in our test of local AI on a VPS with no GPU.
What speed to expect
Coding with a slow model is maddening. Generation depends mostly on memory bandwidth and on how many parameters are read per token, which is why MoE models (which only activate part of themselves) run much faster than dense ones of the same size. Some published measurements with llama.cpp:
| Hardware | Model | Generation | Source |
|---|---|---|---|
| RTX 4090 | gpt-oss-20b | 222 t/s | llama.cpp, discussion 15396 |
| RTX 3060 12 GB (partly in RAM) | gpt-oss-20b, 16,000 tokens of context | 67 t/s | llama.cpp, discussion 15396 |
| Mac M4 Max | gpt-oss-20b | 92.4 t/s | llama.cpp, discussion 15396 |
| Mac M2 Ultra | gpt-oss-120b | 79.7 t/s | llama.cpp, discussion 15396 |
| NVIDIA DGX Spark (128 GB) | gpt-oss-120b | 60.6 t/s | llama.cpp, discussion 16578 |
| RTX 4070 12 GB + 64 GB DDR5 | gpt-oss-120b, 32,000 tokens of context | 25 t/s | carteakey.dev |
For a dense model like Qwen3.6-27B, we couldn’t find a published measurement we could verify. What we can calculate is the ceiling: an RTX 3090 has 936 GB/s of bandwidth and the model takes up about 18 GB, so it can’t go above roughly 50 tokens per second (936 ÷ 18), and in practice it stays below that. That’s still fast for reading, but an agent that generates thousands of tokens per task will feel it.
How to connect it to your editor or to Claude Code
Ollama, LM Studio, and llama-server expose an API compatible with Anthropic’s, so Claude Code can use a local model:
ollama pull qwen3.6
ollama launch claude --model qwen3.6
Ollama’s documentation recommends 64,000 tokens of context or more for working with large repositories. That makes sense: Claude Code sends a lot of instructions plus the contents of the files it reads. Adjust Ollama’s context before blaming the model. If it’s your first time with the tool, our Claude Code guide explains the workflow, and a good CLAUDE.md helps a small model even more, since it needs its context spoon-fed.
For autocomplete in VS Code, Continue and Cline connect to Ollama (port 11434) or LM Studio (port 1234). Aider works with aider --model ollama_chat/nombre-del-modelo.
The math: local vs. the API, in euros
What Claude costs
- Subscription: Claude Pro, $20 a month, includes Claude Code. Max 5x, $100; Max 20x, $200. We break it down in how much Claude costs.
- API, per million input / output tokens: Opus 5.5, $4 / $20; Sonnet 5.5, $2 / $10; Haiku 5.5, $0.10 / $0.50 (with prompts of up to 100,000 tokens).
- Open models via API, for comparison: on OpenRouter, gpt-oss-120b from $0.03 / $0.15 and Qwen3.6-27B from $0.25 / $2.
What local costs
- Hardware. A used RTX 3090 (24 GB) went for around $1,200 in October 2026 according to price aggregators, in a market heavily inflated by AI demand. A Mac Studio with M5 Max starts at $2,499 and goes up to 128 GB of unified memory.
- Electricity. For an average-consumption Spanish household (2,500 to 4,999 kWh a year), a kWh cost €0.2509 in the first half of 2026, taxes included, according to Eurostat.
- Your time to install it, keep it updated, and pick a model. It isn’t zero.
An electricity example, using our own assumptions: a PC with a GPU drawing 400 W total while generating, 4 hours a day of real use, and 22 days a month comes to 35.2 kWh: about €8.80 a month. If the machine stays on idle the rest of the day, add the idle consumption.
The break-even point
With those numbers, a $1,200 RTX 3090 takes:
- About 5 years to pay for itself against Claude Pro ($20 a month), not counting electricity. And it doesn’t do the same as Claude Pro.
- Around a year against Max 5x ($100 a month), if it truly replaced that usage, which for agent work it doesn’t.
- Never, or almost never, against cheap open models via API. If you generate 5 million output tokens a month with Qwen3.6-27B on OpenRouter, you pay about $10 for output plus input. The electricity from the example above already eats up a good chunk of that.
Flavio Copes reaches a similar conclusion in his August 2026 analysis: a $1,800 card pays for itself in about 20 months against the most expensive API, in about 44 against a mid-priced one, and against the cheapest it never pays off.
There’s a third route: renting the GPU by the hour. On RunPod, an RTX 3090 costs $0.22 an hour in the community cloud and an RTX 4090, $0.34. Eight hours a day for 22 days with a 3090 comes to about $39 a month, with no purchase and no noise, and you shut it off when you’re not using it.
Our recommendation
- You code daily and want the best results: Claude, with a subscription. It’s what we use for all our agent work, and no model that fits at home matches it today.
- You already have a 24 GB GPU or a Mac with 32 GB or more: install Qwen3.6-27B or Muse Glimmer-30B and use it for autocomplete, scripts, and anything confidential. It’s free at the margin and takes requests off your subscription.
- Your code can’t leave the company: local stops being a question of price. A Mac Studio or a PC with a 24 GB GPU running Qwen3.6-27B is today the option with the best balance between quality and hardware.
- You only have a regular laptop or a VPS with no GPU: don’t try to code with local AI. The models that fit there fail at the basics, as we saw.
- You want to save without buying anything: first try open models via API or rent a GPU by the hour. If after a month you use them so much that the bill hurts, then do the math on buying.
What to do now
- Write down which coding tasks you do with AI in a normal week: autocomplete, scripts, long agent runs. That decides more than any benchmark.
- If you have the hardware, install Ollama, download Qwen3.6-27B, and redo three real tasks you’ve already done with Claude. Compare the results, not the feeling.
- Check how much you pay today with the Claude pricing guide and compare it with the math above.
- To choose a tool, read Ollama, llama.cpp or LM Studio, and to choose a model for your hardware, the best local AI models in 2026.