Every model card lists a file size. That number is not the amount of RAM you need. The real requirement is the model, plus the conversation's cache, plus the operating system, plus enough headroom that the OS does not start swapping.
Get that sum wrong and a model that “fits” becomes a machine that freezes mid-answer. This guide gives you the sum for 7B, 13B and 70B models, using the same 8 GB mini PC as every other test on this site.
The verdict
8 GB runs a 7B model at Q4 and leaves about 2 GB free on this Ubuntu bench (under 1 GB on Windows) — usable only with everything else closed. 16 GB is the real minimum for a 7B model with a browser open, and the entry point for 13B. 32 GB makes 13B comfortable. 64 GB is where 70B starts, and it is a desktop workload, not a laptop one.
The three things that consume memory
Plan with this formula:
RAM needed = model weights + KV cache + OS + runtime + headroom
- Model weights: the GGUF file. A 7B model at Q4_K_M is roughly 4.1–4.7 GB.
- KV cache: the conversation's working memory. It grows with context length and can add 1–2 GB on a 7B model at 8k tokens.
- OS and runtime: Ubuntu 24.04 with GNOME idles at roughly 1–1.5 GB; Windows 11 at 2.5–4 GB. Ollama or llama.cpp adds a few hundred MB, and a browser adds 1–1.5 GB.
- Headroom: keep at least 1–2 GB free. Below that, one background update is enough to push the model into swap.
Quantization changes the first term dramatically. Storing weights at 4 bits instead of 16 cuts the file to roughly a quarter. That is why a 7B model can run on hardware that would never hold the unquantized version. The trade-offs are covered in GGUF quantization explained.
Model weights by size
Approximate GGUF sizes. Real files vary by model architecture, tokenizer and quant recipe.
| Model | Q4_K_M (4-bit) | Q8_0 (8-bit) | FP16 |
|---|---|---|---|
| 1.7B | ~1.1 GB | ~1.8 GB | ~3.4 GB |
| 3B | ~2.0 GB | ~3.3 GB | ~6 GB |
| 4B | ~2.5 GB | ~4.3 GB | ~8 GB |
| 7B | ~4.1–4.7 GB | ~7.2 GB | ~14 GB |
| 8B | ~4.9 GB | ~8.5 GB | ~16 GB |
| 13B | ~7.9 GB | ~13.8 GB | ~26 GB |
| 30B | ~18–19 GB | ~32 GB | ~60 GB |
| 70B | ~40–43 GB | ~70–75 GB | ~140 GB |
Two things stand out. First, 8-bit roughly doubles the requirement, which is why Q8 is a luxury at 16 GB and a non-starter at 8 GB. Second, the jump from 13B to 70B is not incremental — it is a different class of machine.
The KV cache is the part nobody budgets for
The KV cache stores attention state for every token in the context window. It is why a model that loads fine can still die halfway through a long chat.
Approximate additions, for chat-style use with one sequence:
| Context | 7B | 13B | 70B |
|---|---|---|---|
| 4k | ~0.5–0.8 GB | ~1–1.5 GB | ~3–4 GB |
| 8k | ~1–2 GB | ~2–3 GB | ~5–8 GB |
| 32k | ~4–8 GB | ~6–10 GB | ~20–30 GB |
These numbers move with layer count, head configuration and cache quantization, so treat them as planning ranges. The practical lever is num_ctx: if you do not need 32k of context, do not pay for it. On this machine, keeping context at 4k is the difference between a 7B model that answers and one that swaps.
The OS tax
An 8 GB machine does not give a model 8 GB. On a clean Ubuntu 24.04 boot, this machine reports about 6.5 GB available; plan with about 5.5 GB after a 1 GB safety margin. That is the number to plan with.
| OS | Idle memory | Notes |
|---|---|---|
| Ubuntu 24.04 + GNOME | ~1–1.5 GB | Measured on this bench; browser closed |
| Windows 11 | ~2.5–4 GB | More with browser, Teams, antivirus |
| Linux (light desktop) | ~0.8–1.2 GB | The leanest option for an old laptop |
| macOS | ~4–6 GB | Unified memory; the GPU shares the same pool |
This is also why advice written for 64 GB workstations translates badly. “16 GB is enough for 7B” assumes a lean OS and no browser. On Windows, 16 GB is enough for 7B with a browser — barely.
Recommended RAM by model size
“Minimum” means the model loads and runs with nothing else open. “Comfortable” means a browser, an editor and normal background apps still fit.
| What you want to run | Minimum | Comfortable |
|---|---|---|
| 1B–4B at Q4 | 8 GB | 16 GB |
| 7B–8B at Q4, 4k context | 8 GB (Linux, one task at a time) | 16 GB |
| 7B–8B at Q4, 8k+ context | 16 GB | 24–32 GB |
| 13B at Q4 | 16 GB | 32 GB |
| 30B at Q4 | 32 GB | 48–64 GB |
| 70B at Q4 | 64 GB | 96–128 GB |
If you only remember one row: on Windows, 16 GB is the sensible floor for a 7B model; on a lean Linux desktop, 8 GB is workable for one task at a time. 32 GB is the sensible floor for 13B.
What this looks like on real machines
On the 8 GB test bench, with the browser closed:
| Model | Quant | File | Speed | Experience |
|---|---|---|---|---|
| Qwen3 1.7B | Q4_K_M | 1.4 GB | 10–12 tok/s | Fast, useful for short tasks |
| Llama 3.2 3B | Q4_K_M | 2.0 GB | 6.4–7.1 tok/s | The best balance here |
| Qwen2.5 7B | Q4_K_M | 4.7 GB | 3.2–3.4 tok/s | Works, about 2 GB free |
| Llama 3.1 8B | Q4_K_M | 4.9 GB | 3.0–3.2 tok/s | Slow; little margin |
Those figures come from the full 8 GB test. The pattern matters more than the exact tokens per second: the 7B model is usable for one careful question. What it cannot do is share the machine with a browser. On Windows the margin is under 1 GB; on this lean Ubuntu install it is about 2 GB.
At 16 GB, a 7B model has 4–5 GB free and can coexist with a browser. A 13B model fits at Q4, but generation drops to roughly 1–2 tokens per second and the machine is doing little else. That is the 8 GB vs 16 GB upgrade in one sentence: 16 GB raises the multitasking ceiling, not the speed ceiling.
I have not run a 70B model on this machine, because it does not fit. The math says 64 GB at Q4 before the KV cache, and CPU generation at that size is around or below 1 token per second. Anyone claiming a good 70B chat experience on a laptop is either using a much smaller quant, a GPU, or a remote API.
What to buy, by budget
- Under $60: a matched 16 GB SO-DIMM kit, if your laptop has a slot. This is the single best upgrade for 7B models.
- $100–$200: a used 16 GB mini PC with a desktop CPU and better cooling. Faster than the laptop, not portable.
- $200–$400: 32 GB in a mini PC or an upgradeable laptop. This is the 13B tier, and the point where local models start to feel normal.
- $600+: 64 GB or a GPU. Only worth it if you have a specific 70B or image-generation use case.
Do not buy the label
“AI PC” marketing sells NPUs, and NPUs do not accelerate GGUF CPU inference in Ollama or llama.cpp today. Capacity and memory bandwidth matter far more than an NPU for the models in this guide. Also check whether RAM is soldered: 8 GB of LPDDR4X cannot be upgraded, and no software setting fixes that.
For specific kits and machines that fit this test bench, see the gear page. Prices there are a guide, not a quote.
Check your own machine in two minutes
- Run a model and look at the actual footprint:
ollama psshows the loaded size. - Check the eval rate:
ollama run qwen3:1.7b --verbose "Write 50 words about SSDs." - Watch
free -h(Linux) or Task Manager → Performance → Memory (Windows) while it generates. If committed memory approaches the limit, you are swapping. - Keep at least 1–2 GB free during generation. If you cannot, drop to a smaller model or a lower context.
The rule from the quantization guide still applies: pick the largest model that fits at Q4_K_M with at least 1 GB to spare. Do not squeeze a bigger model into Q2 to hit a number on a spec sheet.
FAQ
Is 16 GB enough for a 7B model?
Yes, comfortably at Q4_K_M with a 4k–8k context and a browser open. It is the configuration I would recommend to most people buying a laptop for local AI today.
How much RAM for a 13B model?
16 GB is the minimum; 32 GB is comfortable. At 16 GB you will fit the weights (~7.9 GB) but leave little room for the KV cache, Windows and anything else. Expect 1–2 tokens per second even when it fits.
Can I run a 70B model on a laptop?
With 64 GB or more, at a low-bit quant, yes — technically. The weights are about 40–43 GB at Q4, so a 64 GB machine has little room for context. Speed is the bigger problem: CPU inference at this size is around or below 1 token per second.
Does more RAM make a model faster?
Not by itself. Speed depends on CPU compute and memory bandwidth. More RAM stops swapping, and dual-channel memory can add roughly 5–15% by increasing bandwidth. Do not expect a 2× gain from capacity alone.
Is 32 GB overkill for 7B models?
Yes. If 7B is all you run, 16 GB is the better spend. Move to 32 GB when you want 13B, or 30B at Q4 with other apps open.
Keep reading
- Can you run an LLM on 8 GB of RAM?
- 8 GB vs 16 GB: is the upgrade worth it?
- GGUF quantization explained
- The best small LLMs for 8 GB
- The full benchmark table