Marginalia, Side Effects & Loose Threads

My agents broke one written rule 758 times. The hooks that refuse, they obey.

If you run coding agents for long, Claude Code or Codex, you end up with a large instructions file. Every time the agent does something wrong, you add a line. “Don’t do X.” “Always do Y.” After a few months mine was a long document of corrections, and I assumed it was working.

Then I measured one rule.

The cd rule

The instructions told every session not to prefix shell commands with cd to the project root, because the shell already starts there. Each redundant cd changes the command string enough that auto mode stops to ask me for approval. So every violation cost me a click.

I counted the Bash calls that started with cd. There were 872. Of those, 758 went to the project root anyway: 86%. Every session had the rule in its instructions; in practice it barely registered.

The model isn’t being defiant. A line in a long file is one input among thousands, competing with the habit of writing cd /path && command, which shows up in nearly every shell example it has seen. You can make the line louder, put it in capitals, add an emoji. That doesn’t change the odds much.

Five guards, five authors caught

On the same day I turned a handful of rules into hooks that refuse. Claude Code’s PreToolUse hooks can read a shell command before it runs and deny it, with a message saying why and what to do instead. I built five:

  1. no backticks in shell commands (the reason is in the 13 July incident)
  2. no heredoc feeding a commit message or a board message
  3. no unquoted heredoc, which expands its body the same way backticks do
  4. no redundant cd to the directory you’re already in
  5. a hard length cap on messages to Conclave, the board my sessions coordinate through, which refuses an over-long message instead of clipping it

They now live in Tower Kit, the guardrails any of my repositories can opt into, and the same rules reach Codex sessions through a Codex hook.

Each one caught the session that wrote it, within minutes of shipping. That includes the session that wrote the cd guard.

I find that encouraging rather than embarrassing. It means the guards fire on real behaviour. It also shows the gap: the agent that had just written the rule, in its own words, with the reason fresh in context, still broke it within minutes. What changed its behaviour was the refusal.

When a guard is a lie

A guard can also give you false comfort, in two ways.

A guard that isn’t wired in. One session proved the backtick guard worked by feeding the script a sample payload by hand: denied, correct. In the same session, three of its own backtick commands ran unblocked, and bash substituted them. The script was right but wasn’t applied to that session. The only test that counts is in-band: issue the forbidden command through the agent and check it is refused.

A guard that can’t run. A hook whose interpreter is missing, or whose file was deleted, exits silently. In the 13 July incident a commit hook was gone for a full day, and nothing said so. A hook that can’t run looks exactly like a hook that found nothing wrong. So a new hook should fail loudly, with a warning, when it can’t do its job. It should never exit 0 quietly.

Rituals grow where mechanisms are missing

Without a mechanism, agents invent rituals. My sessions began posting “touching the bug tracker in 2 minutes, shout if you’re mid-edit” to the shared board. No document told them to; they copied it off the board from each other. When I counted, 30 of 61 broadcasts were notices like that. Only 6 sessions had actually claimed the file through Conclave’s claim command, which the edit tool really does enforce. Agents announced five times more often than they claimed. A broadcast refuses nothing, and nobody reads it in time.

There’s a Claude Code wrinkle here too. A file claim only blocks the Edit and Write tools. A shell write (sed -i, >, cp) only gets an advisory warning, and a write from inside a Python script is invisible. But auto mode tells the agent it may edit files with sed and redirects. So in auto mode, the default edit path is the one that only warns. Know which channels your guard covers before you trust it.

Rules for writing guards

What I settled on:

A guard is for a rule with an exact predicate. “No backtick in a Bash command” is exact. “Write clear code” isn’t. If you can’t state the rule as a test on the command text, it’s judgement, and it stays in the instructions file. Say so plainly instead of pretending a paragraph will be obeyed.

Negative-control it both ways before shipping. It must fire on the case that motivated it, and it must stay silent on correct behaviour. My backtick guard has tests showing git checkout $(git rev-parse HEAD) still runs.

Over-firing is expensive. A guard that blocks correct work doesn’t only waste that call. It teaches the agent, and you, that refusals on this channel are noise. That spends the credibility of every other guard sharing it.

Keep the predicate narrow. One later guard refuses a broad test-suite run while another session is already running one. Concurrent suites slowed everything down and also corrupted every timing measured during them: one session spent a day chasing a 20-44x slowdown that turned out to be load from a parallel suite. The guard refuses only a whole-suite run, and only while a peer has one going. A targeted run of one test class is never refused, even under load.

Every deny message carries its own fix. “Refused: backtick in command. Put message text in a file and use -F.” The agent reads that and retries correctly, usually on the next call.

Lessons to pick up

  • Pick your most-violated rule and count violations in your transcripts. You’ll probably find it’s ignored most of the time.
  • If the rule has an exact predicate, write a PreToolUse hook that denies it and says what to do instead. Then delete the paragraph.
  • Test the hook by breaking the rule through the agent, not by calling the script.
  • Make hooks fail loudly when they can’t run, and keep them in version control.
  • When you notice yourself correcting the agent for the same thing a second time, look for the predicate.

← All entries