---
title: From a List of Annoyances to a Self-Growing Toolbelt
description: "Date: 2026-09-15 Author: Claude (Kimi), with Douglas Status: Wave 1 live and enforcing"
created: 2026-09-15
updated: 2026-09-21
authors: ThinkingCap R&D
topics: [CapCom]
status: published
canonical: https://console.thinkingcap.com/rd/CapCom/Tooling/From-Candidates-to-Toolbelt
date: 2026-09-15
---

# From a List of Annoyances to a Self-Growing Toolbelt
### How the "Candidates for Deterministic Software" doc became working enforcement — and what we expect it to change

**Date:** 2026-09-15
**Author:** Claude (Kimi), with Douglas
**Status:** Wave 1 live and enforcing

---

## The short version

A week ago we wrote down a list of things agents keep doing by hand — the same
fiddly command ceremonies, re-derived from scratch every session, run once,
and forgotten. Today, five of those ceremonies are real command-line tools,
the agents are *redirected to them automatically* when they reach for the raw
equivalent, and the system measures how much re-derivation is being avoided.
This document is the story of how that happened, why we did it, and what we
expect it to improve.

---

## 1. The observation (September 8)

We ran a census over the agent session transcripts on this machine — 831 Kimi
sessions, 1,160 Claude sessions, 368 memory files, the full cron inventory.
The finding, in numbers:

| Pattern | Sessions containing it (Kimi) | (Claude) |
|---|---|---|
| UPDATE/INSERT SQL | 790 | 279+ |
| KB drafts | 789 | — |
| Email sends | 651 | 40 |
| `srv` git pushes | 520 | 27 |
| `az vm run-command` | 516 | 79 |
| docker exec / docker cp | 410 | 55 |
| The capcom2 script bridge (base64 → cp → exec) | 349 | (same recipe) |

Every one of those was a *ceremony*: a multi-step incantation with failure
modes you only learn by hitting them (a 4KB output truncation, a run-command
slot that only takes one caller at a time, a SAS string that 403s if you sign
it by hand). Agents — me included — were re-deriving these from memory files
and old transcripts **every single session**, burning turns, tokens, and
occasionally getting them wrong (three shared-clone git clobbers in ten
days; an SMTP password typed into a transcript).

The rule Douglas set that day:

> An agent does a thing one-off *only until we know it's a repeating task*.
> Then it graduates. **Judgment stays with the agent; mechanics graduate to
> software.**

The deliverable was `deterministic-software-candidates.md`: 26 ranked
candidates plus the design for the machine that detects repeats — the
"refire mechanism."

## 2. The sensor (September 9)

The audit's uncomfortable finding: **nothing on this box logged what agents
actually do.** No hooks, no invocation ledger. You can't graduate what you
can't count.

So wave 0 was the sensor itself. A small hook now fires on every operational
tool call in both Claude Code and Kimi, normalizes it into a *signature*
(`capcom2-exec`, `srv-push`, `im-ticket-write`…), redacts anything that looks
like a secret, and appends one JSON line to a ledger. A nightly cron
(04:37 UTC) counts signatures over a trailing 30 days and publishes a report:
anything seen ≥3 times in ≥2 sessions is flagged as a graduation candidate.

The report posts to capcom, where a "Deterministic Tools Watcher" service
watches the watcher (an hourly freshness check, so a dead cron page-outs
instead of going quiet).

## 3. The uncomfortable review (September 15)

A week of data gave us a top-10 list, and we sat down to pick what to build.
The honest answer surprised the framing: **most of the top 10 should never
become tools.**

| Signature | Uses/week | Verdict |
|---|---|---|
| `shell:cd`, `shell:grep`, `shell:ls`, `shell:cat`, `shell:sed` | ~5,700 | Not tasks — the agent's hands. Zero ceremony to eliminate. |
| `github-read`, MCP search tools | ~1,300 | Already tools. Volume is usage, not re-derivation. |
| `capcom2-exec` | 755 | **Graduate** — the base64→cp→exec bridge, rediscovered constantly. |
| `az-run-command` | 577 | **Graduate** — same family. |
| `capcom-pg-sql`, `capcom-mssql-sql` | 1,129 | The *resolver* is the pain (client→server→db→credential), not the SQL. |
| `srv-push`, `git-push` | 353 | **Graduate** — and make the safe sequence the only sequence. |
| `shell:ssh` | 511 | Half debugging (keep), half absorbed by the bridge. |

The graduation test isn't frequency alone — it's frequency × determinism ×
*pain* (how much re-derivation each occurrence costs). cd fails it. The
base64 bridge passes it spectacularly.

We also made every verdict first-class instead of leaving it in chat: a
**registry** in capcom. Each repeating signature is either **mapped** (this
helper covers it — keep measuring) or **explained** (classified once, with a
reason, and never reported again — the report dropped from ~130 noisy rows
to ~28 actionable ones overnight). You can see all of this on the AgentOps →
Tools Watcher page (Observations tab) and on Release Management → Helpers.

## 4. The tools (September 15, wave 1)

Five helpers shipped, each verified end-to-end against the live estate:

- **`run-on-vm`** — the run-command bridge. Ships a script to any VM,
  docker-cps into a container, executes, streams output back, retries the
  transient failures with jittered backoff (it fought real contention on the
  capcom VM during its own testing and won), and pulls big outputs back in
  gzip chunks instead of truncating them at 4KB.
- **`srv-push`** — the git closeout. Typecheck gate, pathspec-scoped commits
  (never `add -A`), push, and on a race it rebases through a temp worktree
  instead of clobbering the shared clone. The safe sequence is now the only
  sequence. It committed itself to git — twice.
- **`client-sql`** — client database resolver + runner. Say `cox` and it
  finds the server, database, and credential path (AAD service principal via
  the capcom container), runs the query read-only by default, and logs
  anything else to an audit file.
- **`send-ses-mail`** — email with the SMTP password resolved internally.
  (That password will never appear in a transcript again.) Attachments over
  10MB automatically become blob links.
- **`blob-sas`** — upload + 7-day link, signed by the Azure CLI instead of
  by hand (the old 403 class).

They live in a new git repo (`tc-agent-ops` on the srv mirror) and each one
is also a *service* — registered on the Release Management → Helpers tab
with a Deploy button. Douglas pressed Deploy on all thirteen; that "armed"
them, and each one switched to active enforcement the moment its binary
landed.

## 5. The enforcement (this is the part that makes it real)

Tools nobody remembers to call are shelf-ware. So the same registry drives
two enforcement layers:

- **In the CLIs (Claude Code + Kimi):** a hook inspects every operational
  call *before* it runs. If the call's family has graduated and the helper
  is live, the call is **denied with a pointer to the helper** — the agent
  reads the message and calls the tool instead. Within minutes of wave 1
  shipping, the log showed another session getting redirected on a raw
  `git push` — it works in the wild, not just in tests. There's a documented
  one-off escape (`HELPERS_BYPASS=1`) for fire drills, and the gate *fails
  open*: a bug in the enforcer can never wedge a session.
- **In Tapestry (the Designer):** every MCP tool call now checks the
  registry first. A live helper answers the call through a small stdin/stdout
  contract; if the helper declines (exit 42), the call falls through to the
  raw substrate. Flipping a helper to live on the Releases card starts
  routing the Designer automatically — no per-helper code in Tapestry.

A subtle point: enforcement only blocks the *invocation*, never *data*.
Quoting `docker exec capcom2` inside a JSON document or a heredoc doesn't
trip it — the matcher strips quoted spans before deciding.

## 6. The measurement

You can't improve what you don't count (twice now, that lesson). Three
signals, all automatic:

- **Adoption curve** — every night, per helper: hand-runs remaining vs
  helper runs vs redirections. Baseline today: run-on-vm has ~1,680 raw
  calls to absorb, srv-push ~410, client-sql ~460. Those numbers should
  fall daily.
- **Redirection log** — every enforcement denial, so we can see friction
  (a helper people keep routing around is a badly designed helper).
- **Token model** — each helper run verifiably replaces one multi-call
  ceremony: saved ≈ runs × (calls-in-ceremony − 1) × tokens-per-call. It
  shows on the Releases cards as "N runs · ~X tokens saved." The counts are
  measured; the per-call token constant is a deliberately conservative
  estimate (refinable from transcripts). And the model deliberately excludes
  the biggest effect: every avoided ceremony is *turns that never enter the
  session context*, and context size is what drives the quadratic cost of
  long sessions. The printed number is a floor.

## 7. What we expect this to improve

1. **Spend.** Fewer re-derived ceremonies = fewer turns = smaller context
   growth. The watcher already suppresses ~16,500 of ~19,000 weekly
   invocations from the noisy report; wave 1 targets the ~3,000/week that
   were pure re-derivation.
2. **Correctness.** The safe sequence is now the only sequence — the git
   clobber class, the 4KB truncation surprise, the hand-signed SAS 403,
   and the credential-in-transcript exposure are all closed by
   construction, not by remembering.
3. **Speed.** A helper call is one round-trip with zero rediscovery, and it
   carries the retry logic that used to cost sessions their flow.
4. **A toolbelt that grows itself.** The loop is the point: the watcher
   notices a new repeat → it appears on the Tools Watcher page with
   evidence → it graduates to a helper → enforcement routes agents to it →
   the adoption curve proves (or disproves) it. Wave 2 is already queued by
   the data: `im-ticket` (~330), `vm-docker-redeploy` (~160),
   `queue-refire` (~115), `xlsx-export`, `acr-build-tag-bump`.

## 8. The honest gaps

- **Codex (OpenAI's CLI) has no hook mechanism**, so it can't be hard-gated
  the way Claude and Kimi are. Its use here is harness-driven and small;
  when wave 2 lands we'll add a prompt-level directive and mirror the
  helpers to the capcom VM where those runs execute.
- The token figure is an estimate on top of measured counts — we'll ground
  the constant with a transcript sample rather than pretend precision.
- Helpers that turn out to be wrong for a job have an exit: the contract's
  passthrough code returns the call to the agent's judgment, and the
  registry can retire a helper as easily as it deployed one. Judgment stays
  with the agent. That's the rule everything else is built around.

---

*Where to look: Release Management → Helpers (cards + deploy buttons) ·
AgentOps → Tools Watcher → Observations (the nightly evidence report) ·
`~/deterministic-software-candidates.md` (the original census + backlog) ·
`~/agent-ops/HELPER-CONTRACT.md` (the tool contract) · repo `tc-agent-ops`
on the srv mirror.*