Local AI Coding Assistant Setup — Ollama + Cline or Continue (2026)

A local coding assistant gives you most of what GitHub Copilot or a cloud chat tool does — autocomplete, refactors, test generation, multi-file edits — without a subscription, without sending your code anywhere, and without an internet connection. In 2026 the open coding models are good enough that this is a genuinely useful daily setup, not just a privacy experiment.

This guide wires up the two pieces that matter: Ollama to serve the model, and a VS Code extension (Cline or Continue) to drive it. If you have not picked hardware yet, read the best GPU for local LLMs hub first — the short version is that 24GB unlocks the best coding models.

What You Need Before Starting

• Ollama installed and running. If not, follow the Ollama install guide — it takes a few minutes.
• VS Code (or a fork like Cursor or VSCodium). The extensions below are VS Code extensions.
• A capable GPU. 16GB VRAM runs 14B-class coding models and autocomplete well; 24GB unlocks the 30B-class models that make agentic editing shine. See the VRAM requirements guide for the full breakdown.
• A few tens of GB of free disk for the model weights.

Step-By-Step Setup

1. Pull a coding model sized to your VRAM

The single most important choice is matching the model to your VRAM so it stays fully on the GPU. From a terminal:

# 24GB GPU — best all-round local coder (MoE, runs in 24GB at Q4)
ollama pull qwen3-coder:30b

# 16GB GPU — strong 14B-class options
ollama pull qwen3:14b
# or
ollama pull codestral

# Any GPU — fast autocomplete companion model
ollama pull qwen2.5-coder:1.5b

A practical pattern is one chat/agent model (the 30B or 14B above) plus one small autocomplete model (the 1.5B). Autocomplete fires constantly, so a tiny fast model there keeps the editor snappy while the larger model handles deeper requests.

Confirm the model is available:

ollama list

2. Pick your extension: Cline or Continue

Both are free, open source, and talk to Ollama locally. They do different jobs and install side by side:

	Cline	Continue
Best for	Agentic, multi-file tasks; planning and running commands	Inline autocomplete + chat sidebar (Copilot-style)
Interaction	You describe a goal; it edits files and proposes steps	You type; it completes and answers in context
Model fit	Shines with a 24GB-resident 30B coder	Works great with a 14B model + a small autocomplete model

If you want a real “do this across my repo” agent, install Cline. If you want fast inline completion and a chat pane, install Continue. Many developers run both.

3. Connect Cline to Ollama

Install the Cline extension from the VS Code Marketplace, open its settings, and choose Ollama as the API provider:

• API Provider: Ollama
• Base URL: http://localhost:11434
• Model: qwen3-coder:30b (or your pulled model)

Then open the Cline panel, describe a task (“add input validation to auth.py and a test for it”), and let it plan and edit. Review each proposed change before accepting — treat it like a fast junior pair, not an oracle.

4. Connect Continue to Ollama

Install the Continue extension, then open its config (the gear in the Continue sidebar, or ~/.continue/config.json / the newer YAML config). Add Ollama as the provider for both chat and autocomplete:

{
  "models": [
    {
      "title": "Qwen3 Coder (local)",
      "provider": "ollama",
      "model": "qwen3:14b"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Autocomplete (local)",
    "provider": "ollama",
    "model": "qwen2.5-coder:1.5b"
  }
}

Reload VS Code. You now get inline completions from the small model and a chat sidebar backed by the larger one — all local.

5. Tune for speed and context

Two settings decide whether this feels fast or sluggish:

• Keep the model in VRAM. If ollama ps shows your model partly on CPU, it is too big for your card — drop to a smaller quant or model.
• Set a sane context window. 8K–16K covers most edits. Pushing context to 64K+ multiplies the KV-cache VRAM cost and slows every request. Raise it only when a task genuinely needs whole-repo context, and enable any “compact prompt” option your extension offers.

Common Mistakes

• Running a model that does not fit your VRAM. The moment Ollama spills layers into system RAM, generation crawls. Match the model to the card — that is what the local LLM GPU hub tiers are about.
• Using one big model for autocomplete. A 30B model is too slow to fire on every keystroke. Use a small 1.5B–3B model for tabAutocomplete and save the big one for chat and agentic edits.
• Cranking context to the maximum. Long context is expensive VRAM. Default to 8K–16K and only raise it for tasks that need it.
• Expecting frontier-cloud quality on the hardest tasks. Local models are excellent for everyday work but still trail the best cloud models on the most complex agentic problems. Use local for the 90% and reach for cloud only when you hit the wall.
• Skipping review. Agentic tools edit real files. Read every diff before accepting — local or not, the model can be confidently wrong.

What To Do Next

• Add a chat UI for non-coding questions with Open WebUI, which uses the same Ollama backend.
• If you also want retrieval over your docs, set up a local RAG stack.
• Compare runtimes if you are deciding how to serve models: LM Studio vs Ollama.
• Browse current model options in best local LLM models 2026.

Frequently Asked Questions

What is the best local model for an AI coding assistant in 2026?

On a 24GB GPU, Qwen3-Coder-30B (a Mixture-of-Experts model that runs in 24GB at Q4) is the strongest all-round local coding model and the best fit for agentic, multi-file work. On 16GB, run a 14B-class model such as Qwen3 14B, DeepSeek-Coder, or Codestral. For pure autocomplete, a small 1.5B–3B model is fast and good enough.

Which coding extension should I use with Ollama: Cline or Continue?

Use Cline for agentic, multi-step tasks — it can plan, edit several files, and run commands. Use Continue for inline autocomplete and a chat sidebar that feels like Copilot. They install side by side in VS Code and both point at Ollama’s local endpoint, so many people run both.

Can a local AI coding assistant replace GitHub Copilot or Claude?

For autocomplete and many edits, yes — a 24GB GPU running Qwen3-Coder-30B gets close for everyday work and stays fully private and offline. For the hardest agentic tasks and the largest context windows, frontier cloud models still lead. The local trade is privacy, no subscription, and offline use for a small quality gap on the toughest work.

Why is my local coding assistant slow?

Usually the model is too big for your VRAM and is spilling into system RAM, or the context window is set very high. Pick a model that fully fits your VRAM, keep the context window reasonable (8K–16K for most work), and enable any “compact prompt” option in your extension to cut the tokens sent each request.

Does this work without an internet connection?

Yes. Once Ollama and your model are downloaded and the extension is configured to use the local endpoint, the whole loop runs offline. Nothing about your code or prompts leaves your machine, which is the main reason to run a coding assistant locally.