Ollama is a local model runner. It wraps the llama.cpp inference engine in a friendly command-line tool and a small library of models you can pull with one command, then run offline on your own hardware. There is no cloud account and no API key. You download a model once, and after that everything happens on your machine. For anyone who cares about privacy, or just wants to work without a network, that is the entire point. Relic follows the same instinct: its search AI runs on-device, so your data never leaves your computer to be understood.
People who search ollama models are usually asking two things: which one their computer can run, and which one is right for what they are doing. Both answers come down to a few concrete facts about parameters, quantization and memory. Once those click, choosing a model stops being guesswork.
How to read an Ollama model listing
Every entry in the Ollama library, and every tag you pull, encodes three things worth understanding before you download gigabytes of it.
- Parameter count. The size of the model, written as something like 7B, 13B or 70B, meaning billions of parameters. More parameters generally means a more capable model and a larger memory footprint. This number is the single biggest driver of whether a model fits your machine.
- Quantization tag. A suffix like Q4_K_M or Q8_0. This tells you how tightly the weights are compressed. Lower numbers mean smaller files and less memory, at some cost to output quality. Most local setups default to a Q4 variant because it is the sweet spot for fitting a useful model in ordinary RAM.
- Context length. How many tokens the model can hold at once, which sets how much text you can feed it plus how long a reply it can track. A longer context window is handy for large documents, but it also consumes memory on top of the model weights.
When you run a command like the one below, Ollama resolves the tag to a specific quantized file, downloads it, and caches it on disk. If you do not specify a tag, it picks a sensible default, usually a mid-size Q4 build.
Quantization, briefly, because it decides everything downstream
A model’s weights are numbers. Full-precision training keeps them at 16 or 32 bits each, which is accurate but heavy. Quantization stores them at lower precision, commonly 4 or 8 bits, in the GGUF file format that llama.cpp and Ollama use. The trade is straightforward: a Q4 model is roughly half the size of a Q8 one and needs about half the memory, in exchange for a small, usually acceptable, drop in quality. Q8 keeps more fidelity but costs more RAM.
The practical takeaway is that quantization is what lets a large model run on a normal computer at all. A model that would need a data-center card at full precision can fit in a laptop’s memory once it is quantized to 4 bits. This is why the Q4 tag is the workhorse of local inference, and why the smallest model that does your job well is almost always the right pick over a bigger one you have to fight to load.
The RAM math that actually decides what you can run
This is the part worth internalizing, because it turns “will this run?” into arithmetic you can do in your head. For a 4-bit (Q4) model, plan on roughly 0.5 to 0.6 GB of memory per billion parameters, then add a bit more for the context window and overhead. A 7B model at Q4 lands near 4 to 5 GB. A 13B lands near 7 to 9 GB. Scale from there.
Mapping that onto common hardware, as of mid-2026, gives a simple ladder:
- 8 GB machine: comfortable with 7 to 8 billion parameter models at Q4. This covers most laptops and is plenty for chat, drafting, summarizing and light coding help.
- 16 GB machine: reaches the 13 to 14 billion class, which is a noticeable step up in reasoning and coherence for harder tasks.
- 32 GB or more: opens up the 30 billion and up class, where the strongest open models start to feel genuinely capable rather than merely useful.
If you want the same ladder framed as a buyer’s guide by hardware tier and task, our companion piece on the best local LLM works through each rung with specific recommendations.
The model families to recognize
The Ollama library carries a rotating set of models, and version numbers change fast, so it is more durable to think in terms of families than exact releases. As of mid-2026 these are the ones you will run into most, kept deliberately version-agnostic.
- Llama.Meta’s family, long the default general-purpose choice, with broad tooling support and a wide range of sizes. A safe first pull for most people.
- Qwen.Alibaba’s family, known for strong general and multilingual performance and for shipping in a wide spread of sizes, including small ones that punch above their weight.
- Gemma. Google’s open family, notable for capable small models that fit tight memory budgets well.
- Mistral. A family with a reputation for efficient models that deliver good quality per parameter, so they run well on modest hardware.
- Phi.Microsoft’s small-model family, aimed at getting surprising capability out of a low parameter count, which makes it attractive for constrained machines.
- DeepSeek. A family that drew attention for strong reasoning and coding-oriented variants, available in sizes from runnable-on-a-laptop up to large.
Alongside these general chat families, the library also carries dedicated code models tuned for programming and dedicated embedding models whose job is not to chat but to turn text into vectors for search and retrieval. Reach for the right category rather than forcing one model to do everything.
Choosing a model by task
General chat and writing
Pull a well-known general model in the largest size your RAM ladder allows, at Q4. On an 8 GB machine that means a 7 to 8B model; on 16 GB, step up to the 13 to 14B class. This is the right default for questions, drafting, summarizing and brainstorming.
Coding
Use a family or variant tuned for code, and lean toward the biggest one you can hold, since coding rewards capability. Test it on your own snippets before trusting it, because coding quality varies more between models than chat quality does.
Embeddings and search
For retrieval-augmented generation or semantic search, pull a dedicated embedding model. These are small, fast, and produce the vectors that let you find text by meaning rather than exact words. They complement a chat model rather than replacing it. This is exactly the class of model Relic uses locally to make everything you copy searchable by meaning.
The commands you will actually use
Ollama’s day-to-day surface is small. These five cover almost everything.
A couple of these are worth a note. ollama ps is how you check what is actually loaded and how much memory it is using right now, which is the fastest way to diagnose a machine that has slowed to a crawl. And ollama rm matters more than it looks, because models are large and it is easy to accumulate a dozen of them and quietly fill a drive.
Honest limits versus cloud models
Local models are genuinely useful, and for privacy-sensitive work they are the only sane choice. The trade is real. A model you can run on a laptop is, as of mid-2026, smaller than the largest hosted models, so on the hardest reasoning, long-context and coding tasks a frontier cloud model will usually still be sharper. You are also doing the work with your own compute, which means slower generation on modest hardware and a warm laptop.
What you get in return is real: nothing you type leaves the machine, there is no per-token bill, and it works on a plane with the wifi off. For a large share of everyday tasks, a well-chosen local model is plenty, and the privacy comes built in. The skill is matching the model to the job and to the memory you have, which is exactly the math this guide is built around.
Frequently asked questions
How much RAM do I need to run Ollama models?
As a rough rule for a 4-bit (Q4) model, budget about 0.5 to 0.6 GB of memory per billion parameters, plus a little extra for the context window. That puts an 8 GB machine comfortably around 7 to 8 billion parameter models, a 16 GB machine around 13 to 14 billion, and 32 GB or more into the 30 billion and up class. A GPU with its own VRAM changes the math, since the model ideally fits in VRAM rather than system RAM.
Where does Ollama store downloaded models?
Models pulled with ollama pull are cached on disk in Ollama's models directory (under your home folder by default, overridable with the OLLAMA_MODELS environment variable). They are stored once and reused across runs, so pulling the same tag again is instant. Run ollama list to see what is on disk and ollama rm to delete a model you no longer want.
What does the quantization tag like Q4_K_M mean?
Quantization shrinks a model by storing its weights at lower precision. The number is the approximate bits per weight, so Q4 is about 4-bit and Q8 is about 8-bit. Lower bits means a smaller file and less memory at some cost to quality. The K_M suffix refers to the specific GGUF quantization scheme. For most local use, a Q4 variant is the common default because it fits far more model into the same RAM with a modest quality hit.
Is Ollama free, and does it send my data anywhere?
Ollama is free and open source, and inference runs entirely on your own machine. The only network activity is downloading a model the first time you pull it. After that you can run fully offline, which is the whole appeal for private or sensitive work.
Which Ollama model should I start with?
Start with a well-known general model in the 7 to 8 billion parameter range at Q4, since it fits most laptops and covers chat, drafting and summarizing well. If you mainly write code, pick a model family tuned for code at the largest size your RAM allows. For search or retrieval, pull a dedicated embedding model rather than a chat model.