How to Run AI Models Locally with Ollama
Get up and running with local AI in under 10 minutes. This step-by-step guide covers installing Ollama on your Mac, downloading your first model, and generating your first AI-powered response, all completely private and offline.
What is Ollama?
Ollama is a free, open-source tool that makes running large language models (LLMs) locally incredibly simple. Think of it as the “Docker for AI models”: it handles all the complexity of downloading, configuring, and running AI models on your own hardware.
With Ollama, you can run popular models like Llama, Mistral, Qwen, and many others directly on your Mac: no internet connection required after initial download, no API keys, and no subscription fees.
System requirements
Minimum requirements
- macOS 11 Big Sur or later
- Apple Silicon Mac (M1 or newer)
- 8GB RAM (16GB recommended)
- 10GB free storage
Recommended for best experience
- macOS 14 Sonoma or later
- M2, M3, or M4 Mac
- 16GB+ RAM
- 50GB+ free storage
Note: While Ollama primarily targets Apple Silicon, experimental Intel Mac support exists but with significantly reduced performance. For the best experience, we strongly recommend using an M-series Mac.
Installation steps
- 01
Download Ollama
Visit the official Ollama website and download the macOS version, or use the command line:
curl -fsSL https://ollama.com/install.sh | shThe install script will download and install Ollama to
/usr/local/bin - 02
Verify installation
Open a new Terminal window and verify Ollama is installed correctly:
ollama --versionYou should see the version number, like
ollama version 0.3.0 - 03
Start Ollama service
Ollama runs as a background service. Start it with:
ollama serveKeep this terminal window open, or run it in the background. On macOS, Ollama typically auto-starts after installation.
Downloading your first model
Ollama makes downloading models as simple as running a single command. Let's start with Qwen3.5 4B, an efficient current-generation model that works well on most Macs.
Download Qwen3.5 (4B)
This downloads the 4 billion parameter version (~2.5GB). Progress will be displayed during download.
Other great starter models to try:
| Command | Description |
|---|---|
| ollama pull qwen3.5:9b | Excellent 9B model for coding and chat (~5.5GB) |
| ollama pull mistral:7b | Popular, well-tested model (4.1GB) |
| ollama pull gemma4:e2b | Google's efficient small multimodal model (~2GB) |
You can browse all available models at ollama.com/library.
Running the model
Interactive chat mode
Start an interactive chat session with your downloaded model:
ollama run qwen3.5:4bYou'll see a prompt where you can type messages. Press Ctrl+D or type
/byeto exit.Single prompt mode
Send a single prompt and get a response:
ollama run qwen3.5:4b "Explain quantum computing"Using the API
Ollama provides a local API for building applications:
curl http://localhost:11434/api/generate -d '{"model": "qwen3.5:4b", "prompt": "Hello!"}'
Essential commands
| Command | Description |
|---|---|
| ollama list | Show all downloaded models |
| ollama pull <model> | Download a model |
| ollama run <model> | Run a model (downloads if needed) |
| ollama rm <model> | Remove a model to free space |
| ollama cp <src> <dst> | Copy a model |
| ollama show <model> | Display model information |
| ollama ps | Show running models |
Troubleshooting
Model downloads are slow
Downloads happen directly from model hosts. Try using a VPN if your connection is slow, or download during off-peak hours. You can also resume interrupted downloads by running the pull command again.
Out of memory errors
Your Mac doesn't have enough RAM for the model you're trying to run. Try a smaller model (use 3B instead of 7B) or close other applications to free up memory. Models require roughly 1GB RAM per billion parameters.
Ollama command not found
The installation directory might not be in your PATH. Add /usr/local/bin to your PATH or restart your terminal. You can also try reinstalling using the official installer from ollama.com.
Connection refused errors
The Ollama service isn't running. Start it with ollama serve in a separate terminal window, or check if it's running with ollama ps.
Next steps
Congratulations! You now have local AI running on your Mac. Here are some ways to expand your setup:
Get told when a better model fits your MacBook Air M5 16 GB
Get one email when a new open-weight model is a better fit for this machine.
For: MacBook Air M5 16 GB
You'll also get the weekly ModelFit email. By signing up you agree to our Privacy Policy. Unsubscribe from either email anytime, in one click.