# Why I moved my 35kb prompts to my own hardware > A security veteran explains why he pulled his agent sessions off frontier APIs, and what broke when he ran them on local Ollama. - Title: Why I moved my 35kb prompts to my own hardware | Shortify - Summary: A security veteran explains why he pulled his agent sessions off frontier APIs, and what broke when he ran them on local Ollama. Your agent transcripts are… - Keywords: self-hosted ai, ollama, prompt engineering, privacy, security, technology, startups, moved, 35kb, prompts, own, hardware - Source: Patrickmccanna — https://patrickmccanna.net/notes-on-migrating-large-prompts-away-from-anthropic-openai-to-self-hosted-llms - Published: 2026-09-13T20:24:47+00:00 - Read time: 3 min - Topics: self-hosted ai, ollama, prompt engineering, privacy, security, technology, startups ## Your sessions are the real data You may guard your files from LLM providers while handing over something better: the transcripts of how you think through your hardest problems. > Your agent sessions are transcripts of the hardest problems you work on. What would it cost you if someone had copies of them? ## The Navier-Stokes moment Public drama over a claimed math solution put a spotlight on providers training on user activity, and the affected mathematicians raised ethics concerns. ## Cannot rule it out When lawyers reviewed the situation, the strongest defense was that theft of user insight could not be ruled out. Nobody can audit the retention or training pipeline, apparently not even the provider. > We can't audit their retention or their training pipeline. Apparently neither can they. ## Not partners, pirates If privacy actually matters to you, the argument goes, running inference on someone else's hardware is the whole problem. Only your own machines give verifiable protection. > These people should not be considered partners. They are pirates. ## Security theater from the top Providers alarm the public about AI beating security controls they never built, while their own researchers run unsandboxed fleets of agents. > Somebody fetch me my fainting couch. ## What real security sounds like The honest claim defenders make is that attacks become expensive enough not to be worth doing. That comes from paying top talent on both the mitigation and exploitation sides. > we've found ways to make it "Pretty Hard" for attackers ## Exploitability grief About three months into serious study, learners break into something they thought was beyond them and it terrifies them. That fear is accurate, and it is competence arriving, not imposter syndrome. > Competence is knowing enough about a technical domain that you can distinguish what you know well from what you need to learn more about. ## Exploitable is not emergency Vulnerabilities are everywhere and always have been. An agent finding one is not a research breakthrough, it is forty years of known work done faster. > Vulnerabilities are legion. ## Refusals hurt defenders Safety filters that block anything hacking-adjacent stop people from finding and fixing flaws in their own systems. > Your safety filters prevent defenders from discovering vulnerabilities because doing so is "hacking" related. ## Vuln hunting in CI The goal is agents that prove exploitability, since defenders cannot tolerate false positives. That needs models willing to run security tests during development and in pipelines. > We need models that aren't averse to the C-word. ## The local rig The experiment runs on a 128GB AMD Ryzen AI MAX+ 395, with 32GB held for the host OS and the rest given to inference, testing abliterated 27b open weight models. ## Big prompts thrash locally Prompts that ran clean on a frontier API fell apart within three minutes. The agent repeated tool calls, re-read files, and rewrote finished work. > Large prompts founder and thrash. ## Context, not model size The smaller model was not the problem. A 35kb prompt eats 14% of a 65k context window immediately, and session history fills the rest fast. > the pre-prompt is basically briefing a man who is reincarnated every ninety seconds ## Single objective prompting Split preprompts into one problem and one resolution each. In opencode, agents become declarative files in ~/.config/opencode/agents rather than scrappy shell invocations. ## Tuning for small windows Raise Ollama's tiny default context explicitly, log session state to disk for frequent handoffs, re-read only the slice you need, cut tool calls per step, and swap "don't do X" for "only do Y". ## Watch for forgetting Track the signals of context exhaustion: back to back identical tool calls, repeat reads of one file, agents restating their objectives, tool-call parse failures, and high turn counts with few file changes. ## The Faustian bargain Fat context windows give chain of thought room to paper over sloppy prompts, so you never learn they are sloppy. That dependency is the product, and it only runs on their hardware. > With fat context and CoT, even bad prompts produce good results. ## The dragon and the coin Providers are cast as Smaug, calm on their hoard until they see gold in your hand. They keep warning you they are dangerous, so pick your threshold for going self hosted. > What threshold is left to be crossed before you start putting your efforts into becoming self hosted? ## Key takeaway Your agent transcripts are the asset, not your data, and self-hosting them means rebuilding prompts as small single-objective units that fit a 65k context window.