The Best Small LLMs for 8 GB RAM, Tested on a CPU-Only Laptop

“Which model should I run?” is the wrong first question on an 8 GB machine. The right one is “which model can I run and still use the laptop?”

I loaded eight small models on the test bench and used each for the same three tasks: rewrite a 120-word email, explain a 40-line Python function, and answer a factual question with a short justification. These are the ones worth your disk space.

The comparison

All speeds are generation-only, measured over five runs with the browser closed. “First token” includes prompt processing, which is where a slow CPU hurts most.

Qwen3 1.7B — the default choice

This is the model I recommend first, and the one that stays loaded on my machine. It answers fast enough to keep a thought moving: a two-sentence reply arrives in about a second, a paragraph in three or four.

Its limits show up in the same places as every small model: it will confidently state a wrong date, lose the thread after a long conversation, and produce generic advice if you ask a vague question. But for the unglamorous work — renaming variables, tightening a paragraph, turning notes into bullets — it is genuinely useful.

Use it for: chat, rewriting, short summaries, brainstorming names, explaining an error message. Do not use it for: math with several steps, citing sources, anything where being wrong is expensive.

ollama pull qwen3:1.7b
ollama run qwen3:1.7b

Llama 3.2 3B — the quality step up

Doubling the parameters buys noticeably better instruction-following. It stays on topic longer, handles “don't use bullet points” more reliably, and is less likely to invent a plausible-sounding answer.

The cost is latency. The first token takes around eight seconds, and a 150-word answer takes about twenty. That is fine if you are drafting something and can look away; it is frustrating if you are iterating on a sentence.

Use it for: drafting emails, summarizing a page of text, answering a question you will check anyway. Skip it if: you mainly want fast, short exchanges.

Gemma 3 4B — the writer's pick

Gemma's prose is the most natural of the group at this size. It follows formatting instructions well and produces fewer of the tell-tale “As an AI language model…” hedges. On a pure writing task, I would take it over the 3B models.

At 4–6 tokens per second, though, a 200-word answer takes close to a minute. It is a “write the first draft, then edit it yourself” tool, not a conversation partner.

Phi-4-mini — the code explainer

Microsoft's small model is unusually good at reading code and describing what it does. It correctly identified a subtle off-by-one bug in a loop that the other small models missed, and its explanations were structured rather than rambling.

It is less good at open-ended writing and tends to be terse. For a developer who wants a second pair of eyes on a function, that is the right trade.

What about 7B and 8B?

They fit — barely. Qwen2.5 7B uses 4.7 GB, leaving under 1 GB for Windows, the terminal and the model's own KV cache. The first answer takes about 95 seconds, then generation runs at 2–4 tokens per second. A 500-word answer takes over two minutes.

There is a legitimate use case: paste a long document, ask one careful question, walk away. If that is your workflow, a 7B model is worth it. If you want a chat window, it is not.

Llama 3.1 8B is the one model here I would actively avoid on 8 GB. At 4.9 GB it leaves almost nothing, and in testing it triggered paging within two prompts. The fans spin up and the answer arrives slower than you could write it yourself.

How to choose in one minute

• You want a fast assistant for short tasks → Qwen3 1.7B.
• You want better answers and can wait → Llama 3.2 3B.
• You care most about writing quality → Gemma 3 4B.
• You want help reading code → Phi-4-mini.
• You need to process a long document once → Qwen2.5 7B, then unload it.
• You want autocomplete in your editor → Qwen3 0.6B or a cloud completion tool; see the coding assistant report.

The settings that matter more than the model

1. Quantization: Q4_K_M unless the model is tiny (then Q8 is fine).
2. Context length: keep it at 4k–8k. A 32k context allocates a large KV cache and eats your headroom.
3. Threads: set to physical cores (4 here), not logical threads (8). Oversubscribing hurts.
4. Keep-alive: unload the model when you switch to a browser-heavy task.
5. Power: “Best performance” and AC power. Battery mode can halve generation speed.

FAQ

Can I run two models at once?

Not usefully. Two 2 GB models plus the OS exceed 8 GB, and both will slow down. Load one, unload it, load the next.

Is a bigger model always smarter?

At the same quantization, generally yes — but instruction tuning and training data matter more than size at this end of the market. A well-tuned 3B can beat a poorly tuned 7B on real tasks.

Do I need a GPU for any of these?

No. These numbers are all CPU-only. A GPU would be much faster, but none of these models requires one.

How do I unload a model?

ollama stop qwen3:1.7b, or wait for the keep-alive timer. Closing the terminal does not always unload it.

Related reports

• Can you run an LLM on 8 GB? The memory math
• GGUF quantization, explained without maths
• Ollama vs llama.cpp: which is faster here
• Full benchmark table with RAM headroom