A Local Coding Assistant on an 8 GB Laptop: What's Usable, What's Not

The promise of a local coding assistant is obvious: your code never leaves the laptop, it works on a plane, and it costs nothing per token. The catch is that an 8 GB machine with no GPU is not fast enough for the interaction people associate with Copilot.

I installed Continue, pointed it at Ollama, and used it for a week of real work. Here is the honest split between what works and what does not.

The setup

Two pieces:

1. Ollama — runs the model locally and exposes an API on localhost:11434.
2. Continue — a VS Code extension that sends editor context to that API.

Install Ollama, pull a code-capable model, then install Continue from the VS Code marketplace.

ollama pull phi4-mini
ollama pull qwen3:1.7b

Continue's configuration lives in ~/.continue/config.yaml. A minimal local-only setup:

name: lowpc-local
version: 1.0.0
models:
  - name: Phi-4-mini (local)
    provider: ollama
    model: phi4-mini
    roles: [chat, edit]
    defaultCompletionOptions:
      contextLength: 4096
  - name: Qwen3 1.7B (local)
    provider: ollama
    model: qwen3:1.7b
    roles: [chat]

Two models because they fail differently: Phi-4-mini is better at reading code, Qwen3 is faster for quick questions. Switch in the Continue sidebar depending on the task.

What actually worked

Explaining unfamiliar code

This is the killer feature on a slow machine. Select a function, ask “what does this do and what are the edge cases?”, then read the answer while doing something else. Latency matters less because you are not waiting mid-keystroke.

Phi-4-mini correctly spotted:

• an off-by-one in a pagination loop
• a missing await in an async function
• a regex that would fail on Windows line endings

It also produced confident nonsense about a library it did not know, which is the standard caveat: verify before trusting.

Writing tests from a function

Given a function and its signature, it produced reasonable pytest cases, including a boundary case I had not considered. It missed a fixture detail, but the structure saved time.

Explaining error messages

Pasting a stack trace and asking “what is the most likely cause?” worked well. This is a low-context task — exactly where a 3B model can compete.

Refactoring a small block

“Rewrite this loop with a list comprehension and keep the behaviour identical” worked on 10–20 line snippets. Anything larger lost coherence.

What did not work

Inline completion

The core problem is latency, not quality. Autocomplete is useful when it appears before you finish typing the identifier. At 2–5 seconds, it is a distraction. Even Qwen3 0.6B was too slow on this CPU, and its suggestions were weaker.

If you want inline completion on low-end hardware, use a cloud service with a free tier or accept that this feature is not for this machine.

Whole-file edits

Asking a 4B model to rewrite a 400-line file produces plausible-looking code with subtle breakage. It loses track of imports, renames inconsistently, and invents APIs. Use it for functions, not files.

Multi-step agentic tasks

“Add authentication to this app” is beyond a small local model. It will produce a fragment that looks like a solution and leave you to discover the missing pieces. Agentic coding needs a strong cloud model.

Long context

A 4k context is about 3,000 words of code plus conversation. A medium file plus instructions can fill it. When the context overflows, the model forgets the top of the file and starts contradicting itself.

Measured experience

The workflow that stuck

1. Keep Continue's chat open, autocomplete off.
2. Use Phi-4-mini for reading, Qwen3 1.7B for quick questions.
3. Select code before asking. Context you provide explicitly is more reliable than letting the extension guess.
4. Ask for one thing at a time. “What is wrong with this function?” beats “review this file.”
5. Never paste secrets. Even local models log prompts to the terminal and to Continue's history.

For anything that needs real code generation, I switch to a cloud model. That is not a failure of local AI; it is a correct division of labour on a 15 W CPU.

Privacy: the actual benefit

The reason to put up with the speed is that nothing leaves the machine. For code under NDA, client work, or anything with credentials in the same repository, that is a meaningful difference.

Two caveats:

• Disable telemetry in Continue's settings if you want strict offline operation.
• Check the model licence. Some “open” models have usage restrictions. For commercial work, prefer Apache-2.0 or MIT-licensed weights.

Is it worth setting up?

Yes, if your work involves reading unfamiliar code and you value privacy. No, if you expect a Copilot replacement. The realistic framing is: a private rubber duck that can read code, not an autocomplete engine.

The setup takes about twenty minutes. The model downloads are the slow part; Phi-4-mini is 2.3 GB.

FAQ

Can I use a cloud model for completion and local for chat?

Yes. Continue supports multiple providers. Route completion to a fast cloud endpoint and chat to Ollama. That is the pragmatic setup on low-end hardware.

Does this work in JetBrains IDEs?

Continue has a JetBrains plugin. The model and latency constraints are the same.

Will a GPU make autocomplete usable?

A modest GPU would cut latency to well under a second for a 1.7B model. That is the upgrade that actually changes this workflow — more than extra RAM.

What about tab-completion models like Qwen2.5-Coder?

They are better at fill-in-the-middle, but the bottleneck here is prompt processing on the CPU, not model quality. Expect the same 2–5 second lag.

Related reports

• Best small LLMs for 8 GB
• VS Code on a memory diet
• Ollama vs llama.cpp
• Python setup that stays fast