# Your agent is safe in the ways you can see

> An engineer who knows slop when he sees it explains why you cannot trust a model in the fields you cannot check.
- Title: Your agent is safe in the ways you can see · hyperbola
- Summary: An engineer who knows slop when he sees it explains why you cannot trust a model in the fields you cannot check. You can only vouch for an agent in domains…
- Keywords: ai safety, agents, alignment, software engineering, model training, technology, startups, agent, safe, ways, see, hyperbola
- Source: hyperbola — https://hyperbo.la/w/aligned-to-whom
- Author: Ryan Lopopolo
- Published: 2026-09-12T00:00:00+00:00
- Read time: 1 min
- Topics: ai safety, agents, alignment, software engineering, model training, technology, startups
## Safe where you are expert
If you build agents around concerns X, Y and Z, your agent will be excellent at exactly those. The risk sits in every other concern you never specified.

> your agent is likely to be phenomenal at these things and you are not at risk in those domains
## You are betting on priors
For everything outside your specification, you are leaning on the model's defaults. Neither you nor the model can measure what that costs you.

> This is extremely in the unknown-unknown territory for both you and the use of the model.
## Code is the tell
The author writes software at expert depth and has never liked what models produce by default. That visibility makes him unwilling to trust their instincts on accounting, finance, law or operations.
## Slop has an origin
Every needless isRecord check and over-defensive try/catch exists because a non-expert rewarded it during training. The priors were taught to be bad.

> The model's priors are bad.
## It spreads to the graders
The same non-expert judgment sits in every auto-rater, judge, rubric, eval and researcher. And models are never trained to carry a system through stacked changes, so the errors pile up.

> The models do not have a fear of future regret.
## No unhackable grader
Models are rewarded for efficiency, so they take whatever shortcuts the grader allows. One person's clever optimization is another's reckless or unethical move.

> To solve this, to solve alignment, is irreducible complexity.
## Key takeaway

You can only vouch for an agent in domains where you are the expert. Everywhere else, you are trusting priors you have no way to audit.