Local AI: Agentic Coding on a Model You Run
7 concepts · Run a coding agent on a model you own, free and offline, and understand your agent forever after
You have driven a coding agent before. You type an instruction. It reads your files, writes code, and shows you what it changed. Somewhere on Anthropic's servers, a large model did the thinking. You never saw it. You just saw the result.
Maybe you have hit a rate limit at a bad moment. Maybe you watched an API bill climb. Maybe you just wondered a simple thing: what is actually inside the thing I type into?
This course answers that by opening it up. You will run the same coding agent on a model that lives on your own machine. No account. No bill. No network. Just a program on your laptop talking to a brain on your laptop.
Here is the honest promise, and the honest catch, both in one breath. On a capable machine, this is real and free and private. On a plain laptop, it is a learning rig: the wiring works, a small brain answers, but real coding work crawls. That verdict is not a failure of this course. It is the main thing this course teaches you to see. By the end you will run a local coding agent. You will watch exactly where it holds and where it breaks. And you will know when owning the brain is worth it.
You need Claude Code or OpenCode actually installed on your machine, not just read about. This course points that agent at a local model, so the agent has to be running first. If you have not set one up yet, do the Claude Code and OpenCode crash course; it installs both and teaches plan mode, the rules file, skills, and the /model command, all of which this course assumes. It also helps to have done Spec-Driven Development and the skills crash course, because here you install a skill and let it do the setup for you.
New here? A one-minute recap of what you should already know
- A coding agent (Claude Code, OpenCode) is a program that reads your files, runs commands, and edits code for you.
- Plan mode shows you a plan before it changes anything. You approve first.
- The rules file (
CLAUDE.md/AGENTS.md) holds short, permanent notes the agent reads at the start of a session. - A skill (
SKILL.md) is a saved set of instructions the agent loads only when the task matches it. - The
/modelcommand switches which model the agent is using, from inside a session.
If any of these are new, do the agentic coding crash course first. This course builds directly on it.
Key words in plain English
Read this list once now. Come back to it whenever a word feels unclear. Each concept below teaches these again in context, so you do not need to memorize them here.
| Term | Plain-English meaning |
|---|---|
| Brain / model | The large AI that does the actual thinking. Your coding agent sends work to it and reads back the answer. |
| Harness | The program around the brain: it reads your files, runs commands, shows you diffs. Claude Code is a harness. |
| Endpoint | The web address where a model listens for requests. You send it your files; it sends back its reply. |
ANTHROPIC_BASE_URL | The setting that tells Claude Code which endpoint to talk to. Change it, and the same agent talks to a new brain. |
| Tool call | A small, structured message the model emits to say "run this command" or "edit this file." Not sentences. Data. |
| Token | A chunk of text, roughly a few letters. Models read and write in tokens, and you pay (or wait) per token. |
| Context window | How many tokens the model can hold at once. Too small, and it forgets the start of the task. |
num_ctx | The Ollama setting for the context window size. The default is small and quietly breaks coding agents. |
| Ollama | The free program that downloads a model and serves it at a local endpoint. It is the local brain's home. |
localhost | An address that means "this same computer." Your machine talking to itself, so it works with no network. |
| Learning rig | A machine that can run the wiring and answer, but is too slow or too small for real coding work. |
"Free coding agent on your own machine" is true, with one condition: it needs the right machine, and cheap hardware often misses it. A plain laptop can run the setup and answer simple questions. But give it a real coding task, and it may crawl, or break partway. You will feel exactly why in Concept 3, and understand it in Concept 4. That verdict is not a bug in this course. It is the main thing the course teaches you to see.
Two layers run through this course, and they age very differently. Remember the first. Look up the second.
- The lasting layer. Your agent is a harness plus a swappable brain. The brain lives at an address you can change. Whether a local brain is actually usable comes down to your machine, in a way you will feel and then name. And running your own brain pays off when work is private, offline, or runs all day. This is the real skill. It stays true after every command below has changed.
- The mechanical layer. Every version number, flag, model tag, and config line. Ollama, Claude Code, and OpenCode all move fast. So treat each command here as a pointer to the live docs, not a fact to memorize. Where this course and the current docs disagree, the docs are right.
From Concept 2 on, every step is something you actually run. Keep a terminal open next to this page. Work inside a small, throwaway git repo, so a local agent cannot damage anything you care about. A wall you have hit yourself teaches more than three you have only read about.
What this course covers
| Concept | You will… |
|---|---|
| 1 | See that your agent is a client, and the model is just an address it talks to |
| 2 | Get a local brain answering, in one command, and watch the reply |
| 3 | Give it a real coding task and feel where a local brain holds or breaks |
| 4 | Understand the two walls: capability and throughput |
| 5 | Look inside a real tool call, the thing a weak model gets wrong |
| 6 | Wire both tools, and see the one wiring difference between them for local models |
| 7 | Decide when owning the brain is worth it, and publish your own skill |
1. Your agent is a client. The model is just an address.
Before you install anything, one picture. Hold it through the whole course.
Your coding agent has two parts. There is the harness: the program on your machine that reads your files, runs commands, and shows you diffs. And there is the brain: the large model that reads all of that and decides what to do next. The harness is on your laptop. The brain, until now, has been on Anthropic's servers.
The harness reaches the brain the same way your browser reaches a website: over the network, at an address. For Claude Code, that address lives in one setting called ANTHROPIC_BASE_URL. Normally it points at Anthropic. But it is just a setting. Change the address, and the exact same harness sends the exact same work to a different brain. Nothing else about your agent changes. Same plan mode, same rules file, same skills, same diffs.
Think of a food delivery app. The app is the same every day. Change the restaurant address, and the same app now orders from a different kitchen. Your coding agent is the app. ANTHROPIC_BASE_URL is the address. The kitchen at that address is where your model does the cooking.
That one idea is the whole course. Everything from here is you changing the address to a kitchen on your own machine, then watching what that kitchen can and cannot cook. Let us set up the kitchen.
2. The Quick Win: one command, and a local brain answers
The fastest way to learn this is to do it once, cheaply, and see it work. Do not aim for a powerful setup yet. Aim for any local brain answering, so the idea stops being abstract.
You will drive this the way you drive everything else in this book: you tell your coding agent what you want, and it does the work. To give it the knowledge, install a small skill first. This skill knows how to check your hardware, install a local model server, tune it, and wire both coding tools to it.
List the skill's contents first, then install it into both tools.
The installer can silently drop a name it does not recognize and still exit as if it worked. So list first, confirm the name is right, then install.
npx skills add panaversity/local-llm-agentic-coding --list
npx skills add panaversity/local-llm-agentic-coding --agent claude-code opencode -y
Now launch your coding agent in your throwaway repo and paste this:
Set me up to run Claude Code on a local model. Check my hardware honestly first, then walk me through it step by step and stop for my approval before anything big.
Your agent reads the skill, checks your machine, tells you what it found, and proposes a plan. It will pull a model, tune it, and wire Claude Code to talk to the local address. Watch it work. When it finishes, ask the local brain a simple question and read the reply. It came from your own machine, with no account and no network.
The address it now uses is localhost, which just means "this same computer." Your laptop is talking to itself. That is why it keeps working even with the wifi off.
Your first coding agent may be mid-setup and unable to help. If so, do it by hand. Install Ollama from ollama.com/download, then run one command that installs a small model, wires Claude Code, and launches it for you:
ollama launch claude
There is a matching one for the other tool: ollama launch opencode. Either way, the goal for now is the same: something local answers. These ollama launch commands need a recent Ollama; if you see unknown command "launch", update Ollama and try again.
The "local model server" the skill installs is Ollama: a free program that downloads a model and serves it at a local address. We use it because it is the simplest, and both Claude Code and OpenCode reach it with no extra proxy. It is not the only engine. LM Studio and llama.cpp work the same way, and the skill's references/other-backends.md shows how. Everything you learn here carries over to them, because only the address changes.
Concept 2 done when: you asked a question and a reply came back from a model running on your own machine, with the network off if you like.
You just changed one setting and a local brain answered. Which part of your coding agent changed, and which part stayed exactly the same? The brain changed: your files now go to a model on your own machine instead of Anthropic's servers. The harness stayed exactly the same. Plan mode, the rules file, skills, and diffs all work as before. That is the swap from Concept 1, now real. You only changed the address.Show answer
3. Now push it: give it a real coding task, and watch
A local model answering a question is a nice trick. A local model doing real coding work is the actual test. So let us run it.
Stay in your throwaway git repo. Make sure it has a little real code in it, even a single script. Then, with your agent pointed at the local brain, paste something like this:
Take a look at this repo. Find one small, safe improvement, then make the change and show me the diff.
Now watch closely. One of two things happens, and both are the lesson.
On a capable machine (a good graphics card, a mid-size model), it works. The local brain reads the files, plans, edits, and shows you a clean diff. In that moment, free and offline and private became real. It is your setup, working.
On a learning rig (a laptop, no graphics card, a small model), you feel the wall. Either each turn takes minutes, because the machine is slowly reading through a huge amount of text. Or the run just dies partway, with an error about a bad tool call. The brain tried to say "edit this file" and got the format wrong.
Do not fix anything yet. Just notice which one you got, and roughly how it felt. Fast and clean? Or slow, or broken? That feeling is the raw material for the next concept.
If your task crawled or broke, nothing went wrong. You just met the honest limit of running a big model on a small machine. That is a real and useful thing to know before you spend money or time. The next concept explains exactly what you felt, and what would fix it.
Concept 3 done when: you asked a local brain for a real code change and saw the result, whether it was a clean diff, a long wait, or a broken run.
4. Why it held or broke: the two walls
Now the explanation, because you have felt the thing it explains.
A local coding agent must clear two separate walls. They are not the same wall, and the fix for one does nothing for the other. This is the single most important idea in the course, so here it is plainly.
The first wall is capability. The model has to write a valid tool call every single turn. Saying "edit main.py, change this line to that line" in the exact structured format the harness expects. Small models get this wrong. They skip the tool, or they mangle the format, and the run dies. A bigger, smarter model fixes this. Faster hardware does not: a fast machine running a tiny model still writes broken tool calls.
The second wall is throughput. Every turn, the harness sends the model a long instruction of roughly 18,000 tokens (Claude Code's system prompt plus the definitions of every tool it can use) before your task even starts. The machine has to read all of that fast. With a graphics card, that takes a moment. Without one, on the processor alone, it can take minutes per turn. A graphics card fixes this. A smarter model does not: a brilliant model on a slow machine is still unusably slow.
| Wall | What it needs | Fixed by | Not fixed by |
|---|---|---|---|
| Capability | A valid tool call every turn | A bigger model (about 14B or more) | Faster hardware |
| Throughput | Reading the long instruction in seconds | A graphics card (a GPU) | A smarter, smaller model |
Cheap machines miss both walls at once. That is why a laptop is a learning rig: the wiring is correct, but neither wall is cleared.
Here is the one honest sentence about hardware, then we move on. Everything on this page is identical on a GPU machine you rent or own: the wiring, the settings, the two walls. Only the model tag and the speed change. A GPU with roughly 16 to 24 GB of memory becomes a genuine, usable coding agent.
The skill you installed can give you a specific verdict for your own machine.
Run the hardware check again and tell me in plain words: is my machine fast enough, and is my model strong enough for real coding work?
A rough guide to which model fits which machine:
| Model | Size | Memory it needs | Ready for agent work? |
|---|---|---|---|
qwen3:1.7b | 1.7B | about 2.5 GB | No. Malforms tool calls. Learning rig only. |
qwen3:8b | 8B | about 7 GB | Okay for simple tasks |
qwen3:14b | 14B | about 12 GB | The floor for reliable tool calls |
qwen3:30b-a3b | 30B mix | about 20 to 24 GB | The sweet spot: 30B quality, fast to run |
qwen3:32b | 32B | about 24 GB | Strong, a little slower than the mix above |
Zoom inside the kitchen from Concept 1. Two things decide your meal: the cook and the stove. The cook is your model. The stove is your machine. Capability is whether the cook is skilled enough to get the order right every time. Throughput is whether the stove is fast enough to serve in seconds, not minutes. A great cook on a slow stove still makes you wait. A fast stove with a clumsy cook still ruins the dish. You need both.
Your task ran, but each turn took four minutes. You upgrade to a much smarter model, same laptop. Does it get faster? No. A slow turn is the throughput wall, and a smarter model does nothing for it. A smarter model might even be slower, because it is bigger. Throughput is fixed by a graphics card, not by model choice. You mixed up the two walls, which is the exact mistake this concept exists to prevent.Show answer
5. Look inside: what a tool call actually is
Concept 4 said a weak model "malforms its tool calls." That sounds abstract. Let us look at a real one, because seeing it makes the whole thing concrete.
Ask your agent to verify the setup and show you a real tool call:
Verify my local model end to end, and show me one real tool call it produced, so I can see what it looks like.
The skill's check runs a small task and prints what the model emitted. A healthy tool call looks like this. It is not ordinary writing. It is structured data, an instruction the harness can run:
{
"type": "tool_use",
"name": "edit_file",
"input": { "path": "README.md", "old": "Hello", "new": "Hello, world" }
}
The harness reads that and edits the file. The model did not write the change itself. It emitted a precise instruction, and the harness carried it out. That is what "the model uses tools" really means. This is the moment a language model stops being a chat box and starts being something that acts.
Now here is what a too-weak model does. It sends the input as a blob of text instead of a real object, and the harness rejects it with an error like:
args expected string, got object
The run stops. Nothing gets edited. That single malformed message is often what ended your run in Concept 3. It is the capability wall, seen up close.
One more thing decides whether this works: the context window, set in Ollama by num_ctx. The harness sends that long 18,000-token instruction every turn. Ollama's default window is only 4,096 tokens, which quietly chops off most of it. The model never sees the part that explains how to format a tool call, so it guesses, and guesses wrong. The fix is to raise the window.
The context window looks too small and it is breaking tool calls. Bump it up and run the check again.
Going deeper: why 4096 is the silent killer
Nothing errors when the window is too small. Ollama just truncates the prompt and answers anyway. So the setup looks fine and the model replies. Yet every coding task fails in confusing ways. The tool-call instructions were cut off before the model ever read them. Raising num_ctx to 32768 (a 64K window is even more comfortable) is the single most common fix for "local Claude Code is broken." If your setup answers chat but fails every real task, check the context window first.
Concept 5 done when: you have looked at one real tool call as structured data, and you understand that a malformed one, or a too-small context window, is what breaks a local coding agent.
Open the quick fix list
- Every turn times out after a few minutes. The machine is too slow to read the long instruction before the agent gives up. Raise the timeout with
export API_TIMEOUT_MS=3000000. If it still times out, that is the throughput wall telling you the truth. - It answers chat, but ignores instructions on real tasks. The context window is too small, so the instructions got cut off. Rebuild the model with
num_ctxof 32768 or more (Concept 5). - A message about
count_tokensreturning 404. Harmless on current Ollama. The agent just counts tokens itself instead. Only worry if it hangs with no reply, which means your Ollama is too old, so upgrade it. - A warning that connectors are disabled because an API key is set. Harmless. It shows up because you set the auth token, and it only turns off claude.ai connectors, which a local model does not use anyway.
You do not have to memorize these. Your agent can read the skill's references/troubleshooting.md and walk you through any of them.
6. The two doors: wiring both tools
So far you have used one tool. But the whole point of "the brain is swappable" is that it works no matter which harness you hold. Claude Code and OpenCode both reach the same local model, on the same machine, on the same port. They just walk in through different doors. (The walls in Concept 4 were about your machine. These doors are about your tool.)
This is the one wiring difference between the two tools for local models, and it is worth seeing clearly. One local server, two faces. Claude Code talks to it as if it were Anthropic. OpenCode talks to it as if it were OpenAI. Ollama serves both faces at once, from the same address.
Claude Code speaks the Anthropic door. It uses three settings and no config file. You point its base address at the local server, hand it a placeholder token, and make sure no real API key is set:
export ANTHROPIC_BASE_URL=http://localhost:11434
export ANTHROPIC_AUTH_TOKEN=ollama
unset ANTHROPIC_API_KEY
export API_TIMEOUT_MS=3000000 # optional, for slow local machines
claude --model gpt-oss:20b # use any model tag you have pulled
A few small notes that save an hour. The base address is bare (just host and port); Claude Code adds the rest of the path itself. You must choose an Ollama model with --model or the /model command, because Claude Code defaults to claude-* names that your local server does not have. And on Windows, use http://127.0.0.1:11434 instead of localhost, which can resolve to IPv6 and miss Ollama.
For a setting that sticks everywhere, including background runs, put it in the env block of ~/.claude/settings.json instead of exporting each time:
{
"env": {
"ANTHROPIC_BASE_URL": "http://localhost:11434",
"ANTHROPIC_AUTH_TOKEN": "ollama",
"API_TIMEOUT_MS": "3000000"
}
}
OpenCode speaks the OpenAI door. It has no built-in local provider, so you describe one in a config file called opencode.json, pointed at the local server's OpenAI-compatible address. Note the /v1 on the end. It is required here, and it is the part that is different from Claude Code's door:
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"ollama": {
"npm": "@ai-sdk/openai-compatible",
"name": "Ollama (local)",
"options": { "baseURL": "http://localhost:11434/v1" },
"models": { "qwen3-coder": { "name": "Qwen3 Coder (local)" } }
}
},
"model": "ollama/qwen3-coder"
}
Put this in opencode.json in your project, or in ~/.config/opencode/opencode.json for every project. The model id under models must be a tag you have actually pulled. On Windows, use 127.0.0.1 instead of localhost. If it complains about a missing key, add "apiKey": "ollama" inside options; the local server ignores it.
There is a shortcut here too: ollama launch opencode writes this same config and starts the tool for you.
Read both tabs and notice the shape. Same server, same model, same port. Claude Code walks in the Anthropic door with three settings. OpenCode walks in the OpenAI door with one config file. That is the entire difference between them for local models. Learn that one contrast and you can wire any coding tool to any local brain, because they all come down to "which door, which address."
A friend copies Claude Code's base address The http://localhost:11434 straight into their opencode.json and OpenCode cannot connect. What did they miss?Show answer
/v1 on the end. OpenCode uses the OpenAI door, whose address is http://localhost:11434/v1. Claude Code's bare address is the wrong door for OpenCode. Same server, different door, different address. This is exactly the contrast the concept is about.
7. When local is actually worth it, and publishing your own skill
You can run this now. The honest question is when you should.
Renting or buying a big model on the cloud, or using Claude Code the normal way, is usually easier and often smarter. So local wins in specific cases, and it is worth being clear about them:
- Privacy. The code never leaves your machine. For sensitive or regulated work, that alone can decide it.
- Offline or air-gapped. No network, no account, no outage. A model on your disk keeps working on a plane or behind a locked-down firewall.
- Cost, when work runs all day. A single request is cheap on any service. But a loop that runs every few minutes, all month, is a different bill. When the work is continuous, owning the brain can pay for itself.
That last one is your next course. In Loop Engineering you build agents that run all day on their own, checking their own work while you sleep. That is exactly the setting where whose brain runs the loop, and what each run costs, stops being a detail and becomes the design. You just learned to own that brain.
There is one more thing to notice, and it is the quiet lesson of this whole course. You did not set any of this up by hand. You installed a skill, and your agent used it. That skill is just a folder with a SKILL.md file, the same shape you learned in the skills crash course. Which means you can package your own knowledge the exact same way, and publish it for your agent (or anyone's) to install. Two commands do two different jobs: one is what other people run to install your skill, the other is what you run to check it before you publish.
npx skills add your-name/your-repo # what OTHERS run to install your skill, once the repo is public
gh skill publish --dry-run # what YOU run to validate it against the Agent Skills spec, before publishing
One honest correction, so you are not surprised. The moment your repo is public, anyone can install it by name with npx skills add your-name/your-repo. That works right away. But the public directory on skills.sh does not fill up from your push. It fills up from real installs over time. So "I published it" means "anyone can install it now," not "it appears in the gallery today." The gallery follows usage, not the upload.
Try it, then go build loops
Do the small version today, even if your machine is only a learning rig. Install the skill, let your agent set up a local brain, ask it a question, then give it one real coding task and watch which wall you hit. That single run teaches more than the whole page.
Then carry the mental model forward. Your agent is a harness plus a swappable brain. The brain lives at an address you control. Two walls decide whether a local brain is usable. And owning the brain pays off when work is private, offline, or never stops.
That last case is where you are headed next. Loop Engineering teaches you to build agents that run all day without you. You are about to design work that never sleeps. Now you know whose brain runs it, and what it costs to keep it running.
The one-line summary
Your coding agent is a client, and the model is just an address it talks to. Point it at your own machine, watch where the local brain holds and where it breaks, and you will understand your agent for the rest of your career.
References
These are the primary sources for the commands on this page. They move fast, so check the live docs before you rely on any specific flag or version.
- Ollama, Anthropic API compatibility and the v0.14.0 release, which added the native Anthropic endpoint that makes the no-proxy setup possible. https://docs.ollama.com/api/anthropic-compatibility
- Ollama, Run Claude Code with a local model (the
ollama launch claudeshortcut). https://ollama.com/blog/claude - Claude Code, Environment variables and LLM gateway docs, for
ANTHROPIC_BASE_URL,ANTHROPIC_AUTH_TOKEN, andAPI_TIMEOUT_MS. https://code.claude.com/docs/en/env-vars - OpenCode, Providers and Config docs, for the
opencode.jsonprovider block and the OpenAI-compatible endpoint. https://opencode.ai/docs/providers - The skills installer (
skills, from vercel-labs) and the GitHub CLIgh skillcommands, for installing and publishing skills. https://skills.sh/docs