Your sessions are the real data

You may guard your files from LLM providers while handing over something better: the transcripts of how you think through your hardest problems.

“Your agent sessions are transcripts of the hardest problems you work on. What would it cost you if someone had copies of them?”

The Navier-Stokes moment

Public drama over a claimed math solution put a spotlight on providers training on user activity, and the affected mathematicians raised ethics concerns.

Cannot rule it out

When lawyers reviewed the situation, the strongest defense was that theft of user insight could not be ruled out. Nobody can audit the retention or training pipeline, apparently not even the provider.

“We can't audit their retention or their training pipeline. Apparently neither can they.”

Not partners, pirates

If privacy actually matters to you, the argument goes, running inference on someone else's hardware is the whole problem. Only your own machines give verifiable protection.

“These people should not be considered partners. They are pirates.”

Security theater from the top

Providers alarm the public about AI beating security controls they never built, while their own researchers run unsandboxed fleets of agents.

“Somebody fetch me my fainting couch.”

What real security sounds like

The honest claim defenders make is that attacks become expensive enough not to be worth doing. That comes from paying top talent on both the mitigation and exploitation sides.

“we've found ways to make it "Pretty Hard" for attackers”

Exploitability grief

About three months into serious study, learners break into something they thought was beyond them and it terrifies them. That fear is accurate, and it is competence arriving, not imposter syndrome.

“Competence is knowing enough about a technical domain that you can distinguish what you know well from what you need to learn more about.”

Exploitable is not emergency

Vulnerabilities are everywhere and always have been. An agent finding one is not a research breakthrough, it is forty years of known work done faster.

“Vulnerabilities are legion.”

Refusals hurt defenders

Safety filters that block anything hacking-adjacent stop people from finding and fixing flaws in their own systems.

“Your safety filters prevent defenders from discovering vulnerabilities because doing so is "hacking" related.”

Vuln hunting in CI

The goal is agents that prove exploitability, since defenders cannot tolerate false positives. That needs models willing to run security tests during development and in pipelines.

“We need models that aren't averse to the C-word.”

The local rig

The experiment runs on a 128GB AMD Ryzen AI MAX+ 395, with 32GB held for the host OS and the rest given to inference, testing abliterated 27b open weight models.

Big prompts thrash locally

Prompts that ran clean on a frontier API fell apart within three minutes. The agent repeated tool calls, re-read files, and rewrote finished work.

“Large prompts founder and thrash.”

Context, not model size

The smaller model was not the problem. A 35kb prompt eats 14% of a 65k context window immediately, and session history fills the rest fast.

“the pre-prompt is basically briefing a man who is reincarnated every ninety seconds”

Single objective prompting

Split preprompts into one problem and one resolution each. In opencode, agents become declarative files in ~/.config/opencode/agents rather than scrappy shell invocations.

Tuning for small windows

Raise Ollama's tiny default context explicitly, log session state to disk for frequent handoffs, re-read only the slice you need, cut tool calls per step, and swap "don't do X" for "only do Y".

Watch for forgetting

Track the signals of context exhaustion: back to back identical tool calls, repeat reads of one file, agents restating their objectives, tool-call parse failures, and high turn counts with few file changes.

The Faustian bargain

Fat context windows give chain of thought room to paper over sloppy prompts, so you never learn they are sloppy. That dependency is the product, and it only runs on their hardware.

“With fat context and CoT, even bad prompts produce good results.”

The dragon and the coin

Providers are cast as Smaug, calm on their hoard until they see gold in your hand. They keep warning you they are dangerous, so pick your threshold for going self hosted.

“What threshold is left to be crossed before you start putting your efforts into becoming self hosted?”