Run LLMs Locally With Ollama: The Complete 2026 Guide

Want ChatGPT-class AI that runs on your machine — free, private, offline? That’s Ollama. One install, one command, and you’re chatting with an open-weight LLM locally. This is the complete 2026 guide: setup, choosing models, the API, connecting frameworks, and squeezing performance out of your hardware.

How Ollama Works

Ollama is a local daemon that downloads quantized open-weight models and serves them through a REST API on port 11434. Under the hood it runs llama.cpp — and since v0.30, a dedicated MLX engine on Apple Silicon that makes Macs dramatically faster. Nothing you type ever leaves your machine.

Install & First Chat

# macOS / Linux
curl -fsSL https://ollama.com/install.sh | sh   # or download the app from ollama.com

ollama run llama3.3        # downloads the model on first run, then chats

That’s genuinely it. /bye exits the chat; ollama list shows what you’ve downloaded; ollama ps shows what’s loaded in memory. Recent versions also added ollama launch to spin up coding agents (Codex, OpenCode) wired to local models in one command.

Which Model to Pull (July 2026)

Rule of thumb: a Q4-quantized model needs roughly RAM = parameters × 0.6 (an 8B model ≈ 5 GB). Browse the full library at ollama.com/library.

The API

Ollama exposes its own API and an OpenAI-compatible endpoint — so any tool that speaks OpenAI works locally by changing one URL:

curl http://localhost:11434/v1/chat/completions -d '{
  "model": "llama3.3",
  "messages": [{"role": "user", "content": "Explain quantization in one line."}]
}'

# Python, using the standard openai package
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
r = client.chat.completions.create(model="llama3.3",
    messages=[{"role":"user","content":"Hello local AI!"}])
print(r.choices[0].message.content)

Connecting Frameworks

• LangChain: init_chat_model("ollama:llama3.3") — everything from our LangChain A-to-Z tutorial works locally.
• CrewAI: LLM(model="ollama/llama3.3", base_url="http://localhost:11434").
• AI SDK (React): the community ollama-ai-provider plugs into the same streamText pattern from our React 19 + AI guide.

Performance Tips

Sources & further reading: ollama.com · GitHub (150k+ stars) · Ollama blog (MLX engine, ollama launch). Current release line: 0.30/0.31.