Which Ollama model should you run?
Pick by memory first. A 4-bit (Q4) model needs roughly 0.5 to 0.6 GB per billion parameters, so 8 GB of RAM holds a 7 to 8 billion parameter model, 16 GB holds 13 to 14 billion, and 32 GB reaches the 30 billion and up class. Pull it with ollama pull, then run it offline.
Ollama is a local model runner. It wraps the llama.cpp inference engine in a friendly command-line tool and a small library of models you can pull with one command, then run offline on your own hardware. There is no cloud account and no API key. You download a model once, and after that everything happens on your machine. For anyone who cares about privacy, or just wants to work without a network, that is the entire point. Relic follows the same instinct: its search AI runs on-device, so your data never leaves your computer to be understood.
People who search ollama models are usually asking two things: which one their computer can run, and which one is right for what they are doing. Both answers come down to a few concrete facts about parameters, quantization and memory. Once those click, choosing a model stops being guesswork.
How do you read an Ollama model listing?
Every entry in the Ollama library, and every tag you pull, encodes three things worth understanding before you download gigabytes of it.
- Parameter count. The size of the model, written as something like 7B, 13B or 70B, meaning billions of parameters. More parameters generally means a more capable model and a larger memory footprint. This number is the single biggest driver of whether a model fits your machine.
- Quantization tag. A suffix like Q4_K_M or Q8_0. This tells you how tightly the weights are compressed. Lower numbers mean smaller files and less memory, at some cost to output quality. Most local setups default to a Q4 variant because it is the sweet spot for fitting a useful model in ordinary RAM.
- Context length. How many tokens the model can hold at once, which sets how much text you can feed it plus how long a reply it can track. A longer context window is handy for large documents, but it also consumes memory on top of the model weights.
When you run a command like the one below, Ollama resolves the tag to a specific quantized file, downloads it, and caches it on disk. If you do not specify a tag, it picks a sensible default, usually a mid-size Q4 build. The Llama 3.1 library page, for example, lists the sizes that model ships in: 8B, 70B and 405B.
What does quantization do to a model?
A model’s weights are numbers. Full-precision training keeps them at 16 or 32 bits each, which is accurate but heavy. Quantization stores them at lower precision, commonly 4 or 8 bits, in the GGUF file format that llama.cpp and Ollama use. The trade is straightforward: a Q4 model is roughly half the size of a Q8 one and needs about half the memory, in exchange for a small, usually acceptable, drop in quality. Q8 keeps more fidelity but costs more RAM.
The practical takeaway is that quantization is what lets a large model run on a normal computer at all. A model that would need a data-center card at full precision can fit in a laptop’s memory once it is quantized to 4 bits. This is why the Q4 tag is the workhorse of local inference, and why the smallest model that does your job well is almost always the right pick over a bigger one you have to fight to load.
How much RAM does an Ollama model need?
This is the part worth internalizing, because it turns “will this run?” into arithmetic you can do in your head. For a 4-bit (Q4) model, plan on roughly 0.5 to 0.6 GB of memory per billion parameters, then add a bit more for the context window and overhead. A 7B model at Q4 lands near 4 to 5 GB. A 13B lands near 7 to 9 GB. Scale from there.
Mapping that onto common hardware, as of mid-2026, gives a simple ladder:
- 8 GB machine: comfortable with 7 to 8 billion parameter models at Q4. This covers most laptops and is plenty for chat, drafting, summarizing and light coding help.
- 16 GB machine: reaches the 13 to 14 billion class, which is a noticeable step up in reasoning and coherence for harder tasks.
- 32 GB or more: opens up the 30 billion and up class, where the strongest open models start to feel genuinely capable rather than merely useful.
If you want the same ladder framed as a buyer’s guide by hardware tier and task, our companion piece on the best local LLM works through each rung with specific recommendations.
The model families to recognize
The Ollama library carries a rotating set of models, and version numbers change fast, so it is more durable to think in terms of families than exact releases. As of mid-2026 these are the ones you will run into most, kept deliberately version-agnostic.
- Llama.Meta’s family, long the default general-purpose choice, with broad tooling support and a wide range of sizes. A safe first pull for most people.
- Qwen.Alibaba’s family, known for strong general and multilingual performance and for shipping in a wide spread of sizes, including small ones that punch above their weight.
- Gemma. Google’s open family, notable for capable small models that fit tight memory budgets well.
- Mistral. A family with a reputation for efficient models that deliver good quality per parameter, so they run well on modest hardware.
- Phi.Microsoft’s small-model family, aimed at getting surprising capability out of a low parameter count, which makes it attractive for constrained machines.
- DeepSeek. A family that drew attention for strong reasoning and coding-oriented variants, available in sizes from runnable-on-a-laptop up to large.
Alongside these general chat families, the library also carries dedicated code models tuned for programming and dedicated embedding models whose job is not to chat but to turn text into vectors for search and retrieval. Reach for the right category rather than forcing one model to do everything.
Choosing a model by task
General chat and writing
Pull a well-known general model in the largest size your RAM ladder allows, at Q4. On an 8 GB machine that means a 7 to 8B model; on 16 GB, step up to the 13 to 14B class. This is the right default for questions, drafting, summarizing and brainstorming.
Coding
Use a family or variant tuned for code, and lean toward the biggest one you can hold, since coding rewards capability. Test it on your own snippets before trusting it, because coding quality varies more between models than chat quality does.
Embeddings and search
For retrieval-augmented generation or semantic search, pull a dedicated embedding model. These are small, fast, and produce the vectors that let you find text by meaning rather than exact words. They complement a chat model rather than replacing it. This is exactly the class of model Relic uses locally to make everything you copy searchable by meaning.
Which Ollama commands do you need day to day?
Ollama’s day-to-day surface is small. These five, all in the CLI reference, cover almost everything.
A couple of these are worth a note. ollama ps is how you check what is actually loaded and how much memory it is using right now, which is the fastest way to diagnose a machine that has slowed to a crawl. And ollama rm matters more than it looks, because models are large and it is easy to accumulate a dozen of them and quietly fill a drive.
Honest limits versus cloud models
Local models are genuinely useful, and for privacy-sensitive work they are the only sane choice. The trade is real. A model you can run on a laptop is, as of mid-2026, smaller than the largest hosted models, so on the hardest reasoning, long-context and coding tasks a frontier cloud model will usually still be sharper. You are also doing the work with your own compute, which means slower generation on modest hardware and a warm laptop.
What you get in return is real: nothing you type leaves the machine, there is no per-token bill, and it works on a plane with the wifi off. For a large share of everyday tasks, a well-chosen local model is plenty, and the privacy comes built in. The skill is matching the model to the job and to the memory you have, which is exactly the math this guide is built around.
Sources
- CLI reference, Ollama Docs
- Ollama FAQ, Ollama Docs
- Llama 3.1, Ollama Library
- Qwen2.5, Ollama Library