Ollama is the default recommendation for local AI, and llama.cpp is the engine underneath it. So the obvious question: if Ollama is a wrapper, does the wrapper cost you speed on a machine with no headroom to spare?
I ran the same GGUF files through both, measured generation speed, and watched memory use. The answer is more interesting than “use the faster one.”
The verdict
llama.cpp is 2–8% faster and uses 200–400 MB less RAM. Ollama is dramatically easier to live with. On 8 GB, I use Ollama for chat and llama.cpp when memory pressure is the deciding factor.
What each one actually is
llama.cpp is a C/C++ inference engine for GGUF models. It is the reference implementation for CPU inference, with fine-grained flags for threads, context, memory mapping, quantization and sampling. It ships as command-line tools (llama-cli, llama-server) and as a library other apps build on.
Ollama is a product built around llama.cpp: a background server, a model registry, automatic downloads, a Modelfile format, an HTTP API and simple CLI commands. It hides most flags behind defaults and per-model configuration.
That means the comparison is not “two engines.” It is “engine” versus “engine plus convenience layer.”
The measured difference
Same machine, same Qwen3 1.7B Q4_K_M file, browser closed, five runs:
| Metric | Ollama | llama.cpp | Difference |
|---|---|---|---|
| Generation (1.7B) | 12–18 tok/s | 14–19 tok/s | ~5% faster |
| Generation (3B) | 6–9 tok/s | 6.5–9.5 tok/s | ~4% faster |
| First token (3B, short prompt) | ~8 s | ~7 s | ~1 s |
| Idle RAM with model loaded | ~2.4 GB | ~2.1 GB | ~300 MB |
| Cold start | 2–4 s | under 1 s | — |
| Disk footprint (app) | ~1 GB | ~50 MB | — |
The memory gap is the important number. On a machine with 5.5 GB free, 300 MB is 5% of your entire budget. The speed gap is noise by comparison.
What the benchmark does not capture
Ollama's defaults are tuned for convenience, not for 8 GB. Its default context can be larger than you need, and it keeps models loaded for five minutes. Both behaviours cost memory. llama.cpp makes you choose, which is annoying until it saves you.
Where Ollama wins
Setup time. Install, pull, run. There is no build step and no model file hunting.
ollama pull llama3.2:3b
ollama run llama3.2:3b
Model management. ollama list, ollama rm, ollama ps — you can see what is loaded and how much memory it is using. That is genuinely useful on a tight machine.
A local API. Any tool that speaks OpenAI-style HTTP can talk to http://localhost:11434. This is why Ollama became the default backend for local AI apps.
Automatic GPU/CPU selection. On a machine with a GPU it does the right thing without flags.
Model library. Official tags with sensible quantizations, so you do not have to understand GGUF naming to get started.
Where llama.cpp wins
Memory control. You decide the context size, batch size, thread count and whether to memory-map the file. On 8 GB, -c 4096 instead of a default 32k context can be the difference between running and swapping.
llama-cli -m qwen3-1.7b-q4_k_m.gguf -c 4096 -t 4 -ngl 0 -p "Explain mmap in one paragraph."
Startup speed. No server to start. Useful for scripted, one-shot jobs.
Quantization experiments. If you want to compare Q4_K_M against IQ4_XS on the same model, llama.cpp is the direct route.
Lower baseline overhead. On a 4 GB machine, that 300 MB is not a detail.
A built-in server. llama-server gives you an HTTP API with more tuning flags than Ollama exposes.
The flags that matter on 8 GB
| Flag | What it does | Suggested value |
|---|---|---|
-c |
Context size in tokens | 4096 (not 32768) |
-t |
Threads | Physical cores (4 here) |
-ngl |
Layers offloaded to GPU | 0 (no GPU) |
--mlock |
Lock model in RAM | Omit on 8 GB; can cause failures |
--no-mmap |
Disable memory mapping | Usually leave mmap on |
-b |
Batch size | 128–256 for low memory |
Ollama equivalents live in a Modelfile or environment variables:
# Keep models loaded for 30 seconds instead of 5 minutes
OLLAMA_KEEP_ALIVE=30s ollama run qwen3:1.7b
# Limit context per model (Modelfile)
# FROM qwen3:1.7b
# PARAMETER num_ctx 4096
Which should you install?
Install Ollama if:
- you want to be running a model in ten minutes
- you use other apps that need a local API
- you switch between models often
- you value
ollama psshowing you memory use
Install llama.cpp if:
- you are on 4–8 GB and every 100 MB counts
- you want to script one-shot inference
- you are testing quantizations or flags
- you already know your way around a terminal
Install both if: you are me. They coexist fine; just do not run them at the same time with a model loaded.
The workflow I settled on
- Daily chat and writing: Ollama with Qwen3 1.7B, keep-alive at 30 seconds.
- Long document once: llama.cpp with Qwen2.5 7B, context 4096, then quit.
- API experiments: Ollama, because the endpoint is already there.
- “Why is this slow?”: llama.cpp with explicit threads and context, because I can see every knob.
The 5% speed difference never changed a decision. The 300 MB did, twice.
FAQ
Does Ollama use the same GGUF files?
Yes. It downloads GGUF weights and wraps them with a Modelfile. A custom GGUF can be imported with a Modelfile that points at the file.
Can llama.cpp use Ollama's downloaded models?
They live in Ollama's blob store with hashed names. It is possible to copy them out, but simpler to download the GGUF directly.
Is there a GUI?
Several: LM Studio and Jan are the most common. They are convenient, but on 8 GB they add their own overhead. The CLI tools are leaner.
Which is better on battery?
llama.cpp uses slightly less power at idle because there is no server process. During inference, the difference is negligible — the CPU is the load either way.
Related reports
- Best small LLMs for 8 GB
- GGUF quantization explained
- A local coding assistant that does not freeze the editor
- Benchmark table