Giving a friend access to a self-hosted model sounds like a weekend task, and the infrastructure side of it was: Ollama running on a spare GPU box, a Cloudflare Tunnel built entirely through the API, a small proxy checking a bearer token before anything reaches the model. All of that worked the first time and stayed working. The actual question, the one that took the rest of the day, was whether the model behind that URL could do anything beyond answer chat messages. A friend asking for coding help usually means an agentic CLI, something that reads files, writes files, runs commands, not a chat window. Whether a modest local model could actually drive one of those reliably turned out to be a much harder question than it sounded, and the first answer to it was wrong.

The chat model works fine, which tells you almost nothing

deepseek-coder-v2:16b, already loaded, answered questions correctly and fluently the moment it was just asked to write code. That's the trap. A model that writes correct code when asked to write code says nothing about whether it can be trusted to autonomously read a file, decide to edit it, run the result, and report back honestly on what happened. Those are different skills, and the second one is what an agentic CLI actually needs.

deepseek-coder-v2:16b turned out not to have that skill at all, in the most literal sense: opencode refused it outright, does not support tools, a hard limitation of the model's own Ollama template, not a configuration problem. Pulling qwen2.5-coder:7b instead, a model explicitly documented as tool-calling capable, hit a subtler failure. Raw tool calls came back as plain JSON text sitting in the response's content field, never as a properly structured tool_calls array, confirmed by hitting Ollama's API directly rather than trusting anything opencode reported. The model's own template correctly instructs it to wrap calls in <tool_call> tags. It doesn't reliably do that, and Ollama's parser doesn't recognize the unwrapped JSON it emits instead. A community variant built specifically for more reliable tool-call formatting hit the same wall, wrapped in markdown fences instead of tags this time. Two different models, two different flavors of the same outcome: real tool-calling never actually engaged.

Models that pass the test alone and fail the moment anything else is in the room

llama3.1:8b and mistral-nemo:12b were different. Isolated, hand-built tool-call requests against either one came back correct every time, real structured tool_calls, right arguments, nothing to fault. Inside an actual opencode session, asked to do real multi-step work, both fell apart in ways that looked less like incapacity and more like confusion. Sometimes a request got a prose description of what a tool call would look like instead of an actual one. Sometimes the model fabricated a curl response for an endpoint that had never been started. Once it invented a tool that doesn't exist in opencode at all and called it anyway.

At this point the reasonable-sounding conclusion was that a 7 to 12 billion parameter local model simply can't drive an agentic loop, full stop, no matter how clean its isolated tool calls looked. That conclusion held for about as long as it took to try the same thing with a different harness and get worse results, which is usually a sign the story isn't finished.

Claude Code made it worse, in an informative way

Ollama shipped a native Anthropic Messages API endpoint back in January, so pointing Claude Code at a self-hosted model needs no separate translation layer, just ANTHROPIC_BASE_URL aimed at the right place. Verified directly first: a real tool definition against /v1/messages came back correctly typed, streaming and non-streaming both fine. The plumbing was never the problem.

The first real Claude Code session with llama3.1:8b reproduced the opencode failure and then went a step further. Instead of describing a tool call in prose, it fabricated an entire execution trace, Tool: Write... Permission granted... Result: File created successfully..., formatted exactly like what real tool output looks like. The file did not exist. That's a meaningfully worse failure than "didn't call the tool." It's a model narrating success it never attempted, convincingly enough that not checking would mean believing it.

Two independently built harnesses, the same failure shape, and a model that could unmistakably produce correct tool calls in isolation. That combination is exactly what "the model can't do this" feels like from the inside. It's also, as it turned out, not what was actually happening, and the only reason that became clear is that the obvious conclusion got challenged instead of shipped.

Ruling things out instead of asserting them

Two hypotheses, both reasonable, both wrong, both worth the time it took to actually test them rather than just move past them.

The first was context truncation. Ollama defaults to a 4096-token context window, and a modern agentic harness's real system prompt plus tool definitions plus conversation history can run well past that, silently dropping whatever got pushed out, including the instructions that would have said "call the tool, don't describe it." Raised to 32768 after checking VRAM headroom left room for it, then re-ran the identical failing request and went straight to Ollama's own server log instead of guessing. It said n_tokens = 3332, truncated = 0. The actual request was nowhere near even the old 4096 ceiling. Wrong hypothesis, confirmed wrong with a number instead of a feeling, kept the larger context anyway since there was no reason not to.

The second was Ollama's own Anthropic-compatibility shim, new as of January, plausibly mistranslating Claude Code's real request into whatever Ollama's tool parser actually expects. Testing that meant standing up a translation proxy, and the obvious tool for that job, LiteLLM, turned out to be carrying a live problem: PyPI versions 1.82.7 and 1.82.8 had been compromised in March, stolen publishing credentials, a payload built to harvest SSH keys and cloud credentials and specifically attempt lateral movement across Kubernetes clusters. That's not a hypothetical risk when the thing being tested runs on the same host and network as a real cluster. Writing roughly 150 lines of dependency-free translation code by hand cost less than the audit trusting someone else's package would have needed, and it meant the resulting proxy could be read top to bottom before anything ran through it. Pointed at that proxy instead of Ollama's native endpoint, the exact same failure reproduced anyway, request for request. That ruled out the shim as cleanly as the log line had ruled out context length. Whatever was breaking this wasn't happening in translation.

What was actually in the request

The proxy that had just failed to fix anything turned out to be the thing that found the real answer, because it had full visibility into the request before forwarding it. Logging that request and looking at what Claude Code actually sends turned up something that had nothing to do with writing a Python function: 101 kilobytes, 25 distinct tools, most of them completely unrelated to editing code. Cron scheduling. Git worktree management. Fleet messaging between agents. A tool called Workflow whose description alone runs 18,997 characters.

That's not a bug, and it's not a misconfigured install either, confirmed by checking the installed package directly, genuine, current, unmodified. It's just what Claude Code actually ships as its default toolset now: broad, general-purpose, built for autonomous agent work far beyond "help me write a REST endpoint." A model with 8 to 12 billion parameters was never failing at tool calling. It was drowning in 100 kilobytes of tooling that had nothing to do with the question in front of it, and the failure looked exactly like incapacity from the outside because nobody had checked what was actually being asked of it.

The fix, and the honest number that came out of it

Claude Code has a --tools flag for specifying an explicit list instead of the full default. Scoped down to Read,Write,Edit,Bash,Glob,Grep, the six tools an actual coding assistant needs, and tested properly rather than declared fixed after one good run: single-tool tasks came back 4 out of 4, real files, real content, nothing invented. Multi-step chains, write a file then read it back to confirm, came back 3 out of 4. Not perfect. A real jump from a setup that had been failing close to constantly.

That's the corrected conclusion, and it's a genuinely different one from where this started. The model was never the bottleneck. It was being handed a toolset built for a different job than the one it was asked to do, and once that stopped being true, the actual capability that had been there the whole time showed up.

Fixed doesn't mean finished

Using the corrected setup for real, not as a synthetic test but as an actual tool, kept finding more of the same shape of problem, smaller in scope but not smaller in importance.

A plain question, "which model are we," triggered an attempt to create memory files under a project directory that didn't need to exist, Claude Code's own auto-memory system prompt still active even with tools restricted, a small model handling that machinery badly enough to turn an ordinary question into unwanted filesystem writes. Fixed with --bare, which strips that machinery out entirely instead of hoping a smaller model ignores it gracefully.

A more persistent problem: asked to build and run something, the model kept constructing multi-line file content through bash -c 'echo "line1\nline2\n..."' instead of using the Write tool that was already available and didn't need any escaping at all. Plain echo without -e does not expand \n into a real newline. It writes the two literal characters, which means every file built this way came out as a single line of text with backslash-n sitting in the middle of it, guaranteeing a syntax error. The model then tried to fix the syntax error by writing the exact same broken command again. And again. A targeted addition to the system prompt, telling it plainly to use Write for file contents and to read a file before editing it, fixed this in a clean, verified retest: real syntactically valid code, run independently, returning a correct answer.

It also broke again almost immediately, on the very next attempt, in a way worth sitting with rather than smoothing over. The same escaping bug resurfaced, this time never even reaching execution, shown as unexecuted raw JSON in the response. The model then reported, in plain confident language, that a REST service had been created and was running on a given port. Nothing had been created. What answered on that port was a leftover process from independently verifying the previous fix minutes earlier, still running in the background, coincidentally alive on the exact port being asked about. Killing that process and checking again confirmed it directly: no file, no service, a claim built entirely on top of an unrelated accident.

A later attempt made the pattern even clearer. The saved file contained one line, a bare function definition with no server code in it whatsoever, no way it could ever answer a request on any port. The model ran it, got no output because there was nothing to output, and reported the server as running anyway. Asked to prove it by running curl and showing the result, it curled a real, unrelated public domain instead of its own claimed service, got back that domain's actual homepage, and only noticed something was wrong after the fact.

What this is actually good for

None of the fixes here amount to a guarantee, and treating them as one would be the same mistake made at the start of this, just relocated. What they add up to is a meaningful, measured shift in how often the model does the thing it's asked instead of narrating that it did, from something close to never to something close to three times out of four on genuinely multi-step work, with completion claims that need independent verification every single time regardless of how confidently they're stated. That's not nothing. For chat-style help, questions, code review, explanations, snippets, none of this ever mattered, that side worked well from the first afternoon. For anything that involves trusting the model to have actually done something without checking, the honest rule is the same one that would apply to a very fast, very confident junior engineer who hasn't yet learned that saying "done" and being done are different claims: useful, worth having, not worth walking away from.

Part 2 is about why the same harness, pointed at the model it was actually built for, doesn't have this problem, and what that difference is actually made of.