How to Run Gemma 3 Locally — Step-by-Step (2026)

Gemma 3 (from Google) is one of the most efficient open model families for local use — strong quality for its size, with vision support from the 4B model up. It spans an unusually wide range, from a 270M model that runs on a phone-class device to a 27B that needs a serious GPU. This guide picks the right size for your hardware and runs it in one command.

If Ollama is not installed yet, start with our Ollama install guide.

Pick the Right Size for Your VRAM

The Gemma 3 tags and download sizes from the official Ollama library:

Command	Download size	Context	Type	Comfortable VRAM
ollama run gemma3:270m	292MB	32K	Text-only	CPU / any device
ollama run gemma3:1b	815MB	32K	Text-only	CPU / 4GB+
ollama run gemma3:4b	3.3GB	128K	Multimodal	8GB
ollama run gemma3:12b	8.1GB	128K	Multimodal	16GB
ollama run gemma3:27b	17GB	128K	Multimodal	24GB

All default to 4-bit (Q4_K_M) quantization. The 4B and larger models are multimodal (they accept image input) and ship with a 128K context window. Gemma 3 requires Ollama 0.6 or later. For the memory math, see the VRAM requirements guide or the VRAM calculator.

The mapping most people need:

• 8GB GPU or Apple Silicon: gemma3:4b — efficient, multimodal, punches above its size.
• 16GB GPU: gemma3:12b — the quality sweet spot.
• 24GB GPU: gemma3:27b — the best Gemma quality, comfortable on a 24GB card.
• Edge / minimal hardware: gemma3:1b or 270m — genuinely runs on very modest machines.

Step-By-Step Setup

1. Confirm Ollama is current

ollama --version

Gemma 3 needs Ollama 0.6+. If you are on an older build, update it (re-run the installer).

2. Pull and run your size

For a 16GB card:

ollama run gemma3:12b

The first run downloads the model, then drops you into an interactive chat. Gemma 3 is a strong generalist — good for chat, writing, and summarization, with solid multilingual ability.

3. Use the vision models

The 4B, 12B, and 27B models are multimodal — they can describe or answer questions about an image. In a UI like Open WebUI, attach an image and ask about it; the model reads it directly. The 270M and 1B text-only models cannot do this.

4. Confirm it loaded on the GPU

ollama ps

GPU means full speed; CPU or a split means it did not fully fit — drop to a smaller size.

Common Mistakes

• Running an old Ollama version. Gemma 3 needs Ollama 0.6+. Update first if ollama run gemma3 errors.
• Expecting vision from the 1B/270M models. Only the 4B and larger are multimodal; the tiny sizes are text-only.
• Picking 27B on a 16GB card. It is ~17GB before context — it needs 24GB. Use gemma3:12b on 16GB.
• Underestimating the small models. gemma3:4b is genuinely capable for its size and runs on 8GB — a great default for modest hardware or Apple Silicon.

What To Do Next

• Compare Gemma with other families: best local LLM models 2026.
• Try a reasoning model (DeepSeek R1) or a strong coder (Qwen 3).
• Add a chat UI with image support: set up Open WebUI.
• Confirm a size fits your card with the VRAM calculator; need more VRAM? See 16GB vs 24GB.

Frequently Asked Questions

Which Gemma 3 size should I run?

Match it to VRAM: gemma3:4b for 8GB, gemma3:12b for 16GB, and gemma3:27b for a 24GB card. The 4B is a strong, efficient default for modest hardware; the 12B is the quality sweet spot for mid-range GPUs.

Is Gemma 3 multimodal?

The 4B, 12B, and 27B models are multimodal — they accept image input. The tiny 270M and 1B models are text-only. Use a UI like Open WebUI to attach images to the vision-capable sizes.

How much VRAM does Gemma 3 27B need?

About 24GB. The 27B is a ~17GB download at 4-bit and needs room for overhead and its 128K context on top, which is why it fits comfortably on a 24GB card but not a 16GB one.

Can Gemma 3 run on a laptop or CPU?

Yes — gemma3:1b (815MB) and gemma3:270m (292MB) run on very modest hardware, including CPU. They are the easiest “runs on anything” options, trading quality for size.

Do I need a special Ollama version for Gemma 3?

Yes — Gemma 3 requires Ollama 0.6 or later. Update Ollama if an older version fails to pull or run the model.

---

Last updated: June 2026. Model tags, download sizes, context lengths, and multimodal designations reference the official Ollama Gemma 3 library at time of publication. VRAM targets reference our VRAM requirements guide.