The best local AI models in 2026 for your hardware

The right local AI model for you is decided by your machine’s memory, not by whatever list is trending. In October 2026, with 8 GB of RAM and no GPU, the sensible choice is Qwen3.5 4B or Gemma 4 E2B, which we measured ourselves. With a 12-16 GB GPU, gpt-oss-20b. With 24 GB of VRAM or a 32 GB Mac, Qwen3.6-27B (or the new Qwen3.8-27B) and Gemma 4 31B. And with 128 GB of unified memory, gpt-oss-120b. They are all Apache 2.0 licensed: you can use them at your company without asking anyone for permission.

Below you’ll find the reasoning, a table by type of machine, what we measured ourselves on a VPS without a GPU, and what’s available for voice and images. The data is as of October 17, 2026: in local AI, a guide from six months ago is already recommending outdated models.

The short answer, by type of machine#

Memory and speed figures are detailed and sourced further down. If your machine is a VPS without a GPU, we have a whole article with our own measurements.

What decides it: memory, quantization and model type#

Memory rules#

A model has to fit entirely in memory to run at a reasonable speed: in the GPU’s VRAM, in a Mac’s unified memory or, if there’s no GPU, in RAM. The rough math is:

memory needed ≈ model file size + KV cache for the context + 1-2 GB of headroom

The KV cache is what the model stores from the conversation, and it grows with the context. According to Hugging Face, in Llama 3.1 8B it takes up 0.125 GB with 1,000 tokens, almost 2 GB with 16,000 and more than 15 GB with 128,000, as much as the model itself. A model that “supports 256,000 tokens” doesn’t mean your machine can use them.

Quantization: Q4_K_M as a starting point#

Models are released with 16-bit numbers per parameter. Quantizing means storing them with fewer bits: they take up less space and run faster, at the cost of some precision. The official llama.cpp table for Llama 3 8B puts numbers on it:

Q4_K_M is the usual balance: less than a third of the size and a small loss. Below 4 bits, quality drops fast. If you have memory to spare, go for Q5_K_M or Q6_K; if you’re short, a smaller model in Q4 is better than a big one in Q2.

As a size reference in Q4_K_M: an 8B model takes about 4.9 GB, a 14B about 9 GB, a 32B about 19.8 GB and a 70B about 42.5 GB.

Dense and MoE#

• A dense model uses all its parameters for every token. Qwen3.6-27B and Gemma 4 31B are dense.
• A mixture-of-experts (MoE) model only activates part of them. Gemma 4 26B-A4B activates 3.8 billion of its 26 billion; gpt-oss-20b, 3.6 billion of 21 billion; gpt-oss-120b, 5.1 billion of 117 billion.

The whole model still has to fit in memory, but the speed is similar to that of a model the size of the active part. That’s why MoE models are the best option when you have lots of RAM and little or no GPU: llama.cpp can keep the experts in RAM and the rest on the GPU with --n-cpu-moe. That way, a 12 GB RTX 4070 with 64 GB of DDR5 runs gpt-oss-120b at 25-28 tokens per second, according to tests published by carteakey.

Desliza para ver la figura completa

The families that matter in October 2026#

Two warnings about licenses:

• Llama isn’t Apache. Its license imposes conditions (among them, asking Meta for permission above 700 million monthly active users). Meta has moved to the Muse brand, and Muse Glimmer-30B is Apache 2.0.
• “Open” doesn’t always mean commercial. MiniMax M2.7 prohibits commercial use, and Kimi K3 has branding and revenue conditions. Before putting a model in a product, read its license, not the license of the program that runs it.

Llama 3.1 is still the most downloaded model in the Ollama library, with 120.4 million downloads, versus 27.2 million for Gemma 4 and 22.6 million for Qwen3.5. That’s accumulated popularity, not current quality: today, at the same size, any of the new ones beats it.

Recommendations by type of machine#

Laptop or VPS with 8 GB and no GPU#

Models with 2 to 4 billion parameters in Q4_K_M. Qwen3.5 4B (2.74 GB file) is our first choice for writing and explaining; Gemma 4 E2B (3.11 GB) is the most balanced, and Gemma 4 E4B is the alternative if you have a bit of memory to spare. Use them to summarize, classify, extract data or write drafts you’re going to review. Don’t expect them to reason well or to code without supervision.

We measured six on a VPS with 3 vCPUs of a Ryzen 9 5900X, CPU only, with llama.cpp:

Below 2B, the quality is good for almost nothing: Qwen3.5 0.8B told us that quantization “reduces the number of words” the model can generate, which is false. And watch the names: Gemma 4 E2B has 4.65 billion parameters in total (the “E” stands for “effective”) and takes up more space than Qwen3.5 4B. The details of the tests are in what runs on a VPS without a GPU.

16 GB of RAM without a dedicated GPU#

Qwen3.5 9B is the jump in quality: according to Qwen, 82.5 on MMLU-Pro. We couldn’t measure it, but given its size it will run considerably slower on CPU than the 3B and 4B models in our table. gpt-oss-20b needs 14.9 GB with 8,000 tokens of context according to llama.cpp: on a 16 GB machine with the operating system and browser open, it doesn’t fit comfortably.

8 to 16 GB GPU#

• 8 GB (RTX 4060, 3060 Ti): Qwen3.5 9B entirely on the GPU. For something bigger, gpt-oss-20b with some of the experts in RAM.
• 12 GB (RTX 3060 12 GB, 4070): gpt-oss-20b. In the llama.cpp results, a 12 GB RTX 3060 runs it at 67 tokens per second with 16,000 tokens of context, with part of it in RAM. Ministral 3 14B (about 9 GB in Q4) fits entirely.
• 16 GB (RTX 4060 Ti 16 GB, 5060 Ti): gpt-oss-20b entirely, or Devstral Small 2 if you want it for code.

24 GB GPU (RTX 3090, 4090)#

The sweet spot for local AI at home. The dense 27-31B models fit in Q4:

• Qwen3.6-27B (about 18 GB): the best for coding among those that fit, with 77.2% on SWE-bench Verified according to Qwen.
• Qwen3.8-27B: the newest, from August; Qwen reports 90.3 on LiveCodeBench v6 and 89.2 on GPQA.
• Gemma 4 31B: 85.2 on MMLU-Pro. Artificial Analysis gives it 39 points on its intelligence index, versus 31 for Gemma 4 26B-A4B.
• Muse Glimmer-30B (about 17 GB in 4 bits).

With 24 GB and an 18 GB model you have about 5 GB left for the KV cache: enough for conversations, tight for working with entire repositories. We compare it with Claude in local models for coding vs. Claude.

Mac with Apple Silicon#

Unified memory lets a Mac load models that would need an expensive GPU on a PC. Speed depends on the chip’s memory bandwidth: in the llama.cpp tests with a 7B model in Q4, a base M4 generates 24.1 tokens per second, an M4 Pro 50.7, an M4 Max 83.1 and an M5 Max 119.9.

• 16 GB: Qwen3.5 9B or Gemma 4 E4B. Not all the memory is available to the model.
• 32-64 GB: Qwen3.6-27B, Gemma 4 31B or Qwen3.6-35B-A3B, with room for context.
• 96-128 GB: gpt-oss-120b. In the llama.cpp tests, an M2 Ultra runs it at 79.7 tokens per second (and an M4 Max runs gpt-oss-20b at 92.4).

On Mac, Ollama (since version 0.40) and LM Studio can use MLX, Apple’s library, and llama.cpp uses Metal: it’s worth trying both routes. We cover it in Ollama, llama.cpp or LM Studio.

128 GB of memory: Strix Halo and DGX Spark#

Mini PCs with the AMD Ryzen AI Max+ 395 (up to 128 GB of unified memory) and the NVIDIA DGX Spark (128 GB) are designed for this. In the llama.cpp tests, the DGX Spark generates with gpt-oss-120b at 60.6 tokens per second. Mistral Small 4 (119 billion, 6.5 billion active) and Qwen3.5-122B-A10B also fit here.

How to read benchmarks without fooling yourself#

• The figures on model cards come from each vendor, using their own method. They’re useful for comparing models within the same family, but less so across families.
• Artificial Analysis measures every model with the same method. Its index is the closest thing to a fair comparison. Gemma 4, for example: 31B, 39 points; 26B-A4B, 31; E4B, 19; E2B, 15.
• Your task is the best benchmark. Ten real examples of what you’re going to do tell you more than any table.

Vision, voice and images locally#

Vision. Qwen3.5 and 3.6 understand images out of the box, as does the whole Gemma 4 family. If you’re short on memory, MiniCPM-V 4.6 has only 1.3 billion parameters and an Apache 2.0 license.

Speech to text.

• Whisper large-v3-turbo (809 million parameters, MIT) with whisper.cpp or faster-whisper is still the default option. The large model needs about 3.9 GB of RAM in whisper.cpp.
• For Spanish, NVIDIA Parakeet-TDT-0.6B-v3 scores a 3.45% word error rate on FLEURS, with a CC-BY-4.0 license.
• Cohere Transcribe (2 billion, Apache 2.0) also supports Spanish.

Text to speech.

• Piper runs on a Raspberry Pi and has Spanish voices. Its license is currently GPL-3.0.
• Qwen3-TTS (Apache 2.0, 10 languages, clones a voice from 3 seconds of audio) and VoxCPM2 (Apache 2.0, 30 languages) are the newest.
• Be careful with XTTS-v2: its license only allows non-commercial use.

Images. This is where you do need a GPU.

• FLUX.1 [schnell] is Apache 2.0.
• FLUX.2 [klein] 4B (Apache 2.0) needs about 13 GB of VRAM.
• Stable Diffusion 3.5 Medium needs 9.9 GB not counting the text encoders, and its license is free below one million dollars in annual revenue.
• To run them, one option is ComfyUI, which is open source (GPL-3.0).

What to do now#

1. Check how much memory you have: GPU VRAM, Mac unified memory or free RAM.
2. Pick the model for your row in the table at the top and download it in Q4_K_M.
3. Install the tool that suits you: Ollama, llama.cpp or LM Studio.
4. Test it with ten of your own tasks before deciding that “local AI is useless” or that “it’s just as good as ChatGPT.” Neither is true.
5. If you want it for coding, first read how it compares with Claude and what it costs.