Run AI Offline: Complete Guide to Local LLMs Without Internet

Step 1: Install Ollama (requires internet)

Download the installer for your platform. On macOS, download the .dmg from ollama.com. On Linux:

# macOS: Download from https://ollama.com/download
# Linux:
curl -fsSL https://ollama.com/install.sh | sh

# Windows: Download installer from https://ollama.com/download

Step 2: Download models for offline use

Pull every model you might need while you still have internet. Each model downloads once and is stored permanently on your drive.

# General purpose (pick based on your RAM)
ollama pull llama3.3:8b        # 4.9 GB - needs 8 GB RAM
ollama pull qwen2.5:14b        # 9.0 GB - needs 16 GB RAM
ollama pull llama3.3:70b       # 40 GB  - needs 48 GB RAM

# Reasoning (chain-of-thought)
ollama pull deepseek-r1:8b     # 4.9 GB - needs 8 GB RAM
ollama pull deepseek-r1:32b    # 19 GB  - needs 32 GB RAM

# Coding
ollama pull qwen2.5-coder:7b   # 4.7 GB - needs 8 GB RAM

# Fast and lightweight
ollama pull llama3.2:3b        # 2.0 GB - needs 4 GB RAM
ollama pull gemma2:2b          # 1.6 GB - needs 4 GB RAM

Step 3: Verify models are stored locally

Confirm all models are downloaded and check total storage used:

# List all downloaded models
ollama list

# Check storage location
# macOS/Linux: ~/.ollama/models/
# Windows: C:\Users\<username>\.ollama\models\

# Check total disk usage
du -sh ~/.ollama/models/

Step 4: Disconnect and run

Turn off Wi-Fi, unplug ethernet, and test. If Ollama responds, you are fully offline.

# Disable all network connections, then:
ollama run llama3.3:8b

# Try a prompt:
>>> Explain quantum computing in simple terms

# If you get a response, your AI is running offline!

Step 5: Use the API for app integration

Ollama exposes an OpenAI-compatible API at localhost:11434. Any app that supports OpenAI can connect locally:

# Test the API (works offline)
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.3:8b",
  "prompt": "What is the meaning of life?"
}'

# OpenAI-compatible endpoint
curl http://localhost:11434/v1/chat/completions -d '{
  "model": "llama3.3:8b",
  "messages": [{"role": "user", "content": "Hello"}]
}'

Step 1: Download LM Studio

Visit lmstudio.ai and download the installer for macOS, Windows, or Linux. The app is free for personal use. Install it while connected to the internet.

Step 2: Download models using the built-in browser

Open LM Studio, go to the Discover tab, and search for models. The app shows estimated RAM requirements for each quantization level. Download GGUF-format models. For Mac users, MLX-format models are also supported for better Apple Silicon performance.

Step 3: Sideload models (optional)

You can also "sideload" model files downloaded outside the app. Place GGUF files in the LM Studio models directory and they appear automatically. This is useful for transferring models via USB to offline machines.

Step 4: Go offline

Disconnect from the internet. Open LM Studio, select a downloaded model, and start chatting. The app works identically online or offline. Nothing you enter leaves your device.

Step 1: Install GPT4All

Download from nomic.ai/gpt4all for Mac, Windows, or Linux. The installer bundles everything needed. One-click model installation is built in.

Step 2: Download models and enable LocalDocs

Use the built-in model downloader to pull recommended models. Then configure LocalDocs by pointing it at folders on your machine. GPT4All indexes your documents locally for private, offline retrieval-augmented generation (RAG).

Step 3: Disconnect and use

Once models and documents are indexed, disconnect from the internet. GPT4All continues to work. Ask questions about your documents, generate summaries, and analyze content, all without any network access.

Step 1: Install Jan

Download from jan.ai for Mac, Windows, or Linux. The app is free and open-source. It provides a clean chat interface similar to ChatGPT.

Step 2: Download models

Browse and download models like Qwen3, Llama 3, and Gemma directly within the app. You can also load your own GGUF model files from HuggingFace.

Step 3: Go offline

Disconnect from the internet. Jan works identically offline. It also provides an OpenAI-compatible API server, so other apps on your machine can connect to it locally.

1. Package everything on an online machine

# On internet-connected machine:
# Download Ollama installer
curl -fsSL https://ollama.com/download/ollama-darwin -o ollama-installer

# Download all needed models
ollama pull llama3.3:8b
ollama pull deepseek-r1:8b
ollama pull qwen2.5-coder:7b

# Package the models directory
tar -czf ollama-models.tar.gz ~/.ollama/models/

2. Transfer via approved media

Copy the installer and model archive to a USB drive, encrypted external SSD, or whatever transfer medium your security policy allows. Scan the media per your organization's security protocols.

3. Install on the air-gapped machine

# On the air-gapped machine:
# Install Ollama
chmod +x ollama-installer && ./ollama-installer

# Extract models to the correct path
tar -xzf ollama-models.tar.gz -C ~/

# Verify models are available
ollama list

# Run — fully air-gapped
ollama run llama3.3:8b