Not sure which AI model to choose?
This page explains, in plain English, how to pick a local AI model your computer can actually run. No background knowledge needed — and every model name below is a real, currently-downloadable model from the official Ollama catalogue.
The three kinds of “memory” — and why they matter
System RAM is the general-purpose memory every computer has — it's what the “16GB” on a laptop sticker usually means. Programs load into RAM while they run.
GPU memory (VRAM) is the separate, faster memory on a graphics card (NVIDIA/AMD). If your computer has a graphics card, the model would rather live in its VRAM — and you can roughly divide your card's VRAM number by 2 to estimate the biggest sensible billions-of-parameters model in 4-bit form.
Mac unified memory — on Apple's M-series Macs there's no separate VRAM; CPU and graphics chip share one pool (that's the “Memory” figure in About This Mac). macOS lets the graphics side use a generous portion of it, and modern Macs run local AI well for exactly this reason.
Plain rule across all three: a 4-bit model needs roughly 0.6–1GB of memory per billion parameters while running — more than its download size (see next section). Keep 2–4GB spare for the conversation itself.
Download size is not runtime memory
The gigabytes shown during download are the compressed file. Once a model starts, it's unpacked and a workspace for the conversation is reserved — so a “5GB download” typically needs around 6–8GB of actual memory once running. Practical version: leave headroom — if in doubt, pick the model one size smaller than your memory math suggests.
What also uses memory: context length (how much text the model can consider at once — longer = more memory), quantisation (the compression format; 4-bit is the usual download and the smallest sensible trade-off), your graphics hardware (cards accelerate; CPU-only runs work but are slower), and everything else you have open (browsers are memory-hungry).
Conservative starting recommendations
These agree with the model-selection logic on the main setup page and only use verified, currently-downloadable catalogue models:
| Your computer | Start with | Download | What to expect |
|---|---|---|---|
| 8GB RAM (no strong card) | qwen3:4b | 2.5 GB | May work; may be slow or refuse to load. Not practical for coding-agent work. Try the online guide instead if this fails. |
| 16GB RAM / 8GB+ VRAM | qwen3:8b | 5.2 GB | The main beginner model. Reasonable for chat and small coding tasks; agentic/code-edit features still limited vs cloud. |
| 32GB+ RAM / 16GB+ VRAM | qwen3:8b first; then qwen2.5-coder:14b (9.0 GB) or gemma3:27b (17 GB) | — | Start small, confirm everything works, then try bigger coding models. Bigger is not automatically better at agentic tasks. |
| Mac 16GB unified | qwen3:8b | 5.2 GB | Workable starting point; expect modest speed. |
| Mac 27–32GB unified | qwen3:8b → then gemma3:27b (17 GB) | — | Good capability; still verify each model actually loads before building on it. |
Full catalogue listings (check sizes yourself before downloading): qwen3 · qwen2.5-coder · gemma3 · mistral-small · the whole catalogue.
Models we deliberately do not recommend for beginners
- llama4 (any variant): the smallest official download is about 67GB — it will not load on typical beginner machines. Wonderful technology, wrong starting point.
- Huge “100B+” models of any family: same reason. Beginners should get one small model working end-to-end first.
- Anything promising specific speeds: real performance depends on your machine; the honest answer is “try it and see, expect slower than cloud”.
Mixture-of-experts, in one paragraph
Some catalogue models are “MoE”: a large total size with only a small “active” part working per word. The whole model still has to fit in memory — so check the total size against your memory — but they can answer faster than their size suggests. qwen3:30b (19 GB) is a common example in the catalogue. It's a useful thing to know later; not a beginner starting point.
Quick answer, if you just want a starting point
Most modern computers (16GB RAM, or 8GB+ graphics card): download qwen3:8b. 8GB or less: try qwen3:4b with the expectation it may not work well; also consider the online guide. 32GB+: still start with qwen3:8b, then explore qwen2.5-coder:14b or gemma3:27b once the basics work.
Back to the Offline AI setup guides, or browse our UK AI directory to meet the UK companies building this technology.