---
title: From Bad News to Ground Truth
description: "One day of teaching the Loom honesty — 2026-09-19 → 2026-09-20, tapestry-node seamstresses, everything live."
created: 2026-09-20
updated: 2026-09-21
authors: ThinkingCap R&D
topics: [Tapestry]
status: published
canonical: https://console.thinkingcap.com/rd/Tapestry/Bad-news-to-ground-truth
date: 2026-09-20
---

# From Bad News to Ground Truth

*One day of teaching the Loom honesty — 2026-09-19 → 2026-09-20, tapestry-node `seamstresses`, everything live.*

---

## The complaint that started it

> "In the loom she comes back with bad news a lot… 'I can't answer that'… isn't that a signal we can see before the human — to try it again with a better model, and note when that gets a better answer?"

And then, a day later, the refinement that became the whole design:

> "Just because it is streaming doesn't mean we can't hold a message till we have the gist that it's what we expect, then we can stream the rest. And can't we demand an easy-to-see token if the answer will be of a type — put the work onto the LLM? If the answer to my next question means I'm ugly, say UG ahead of the rest, so I'm ready."

Forty-eight hours later, the loom that was failing answered a calendar question by calling a live tool, reading the raw JMAP response, and recognizing **the very bug it had been assigned to fix** — with evidence, on its own.

This is the story of what got built in between.

---

## Act I — Catching the bad news

The quality gate already *classified* failures every turn — surrender, fabrication, narration — but in live mode it could only log them: the tokens were already on the Weaver's screen. So the first build made the classification **act**:

- **Operator deflection detector** — asking the Weaver for tool names, endpoints, routes, or data shapes her own surface provides. Two tiers: strong (fires even after heavy tool use — effort doesn't license asking) and weak (zero tool calls only — a credential ask after real effort is legitimate).
- **Surrender, sharpened** — "I couldn't find it" now *stands down* when a search tool actually ran. An honest empty result is not a surrender.
- **Auto-retry** — a Tailor-voiced nudge re-drives the turn on a **fresh Tally Max route**, the bad verdict already on record steering the bandit. Steering, not blacklisting: *"one bad instance isn't death to the whole model — it's just data."*
- **The pairing is data** — `metadata.auto_retry_of` links every retry to its bad answer, so "did the better route get a better answer?" is a SQL join, not a guess.

## Act II — The UG token (Reply Shape Sentinel)

Every reply must now open with one machine line, ahead of everything:

```
[[SHAPE: result|report|question|blocked|proposal|chat]]
```

The declaration does three jobs at once:

1. **Routing** — the head-hold watermark (`delta → HeadGate → TailBuffer → WebSocket`) holds the stream's head until the shape is known. Trust shapes (`result`, `report`, `chat`) flush and stream. Risk shapes (`question`, `blocked`, `proposal`) are **held whole and verified before one byte renders** — a deflecting question gets *suppressed*: the Weaver sees an honest Tailor placeholder, the original goes to the Fabric, and the auto-retry drives a fresh answer.
2. **Commitment** — a model that must declare first can be caught *lying about what it declared*. `[[SHAPE: result]]` + zero tool calls + "committed and deployed" = **shape fraud**, provable in writing.
3. **Fail-open grace** — no sentinel within 200 chars and the turn runs exactly as before. Models degrade, never deadlock.

### What the live probes taught (11 bait turns, 9 fixes)

| The probe broke | The lesson |
|---|---|
| Bare `[[question]]`, re-declared mid-pivot | Models compress protocols. Parser takes marker *sequences*, last wins |
| `[[SHAPE: question` arriving without `]]` | Split-marker probe fail-opened mid-marker. Three-form prefix probe |
| `couldn't`, `don't` invisible to every gate | **Her apostrophes are curly (U+2019). Every pattern corpus was straight-quote-only** |
| Four deflection phrasings in four turns | Lexicon whack-a-mole is unwinnable → the **hedge arm**: fact-uncertainty + ends-with-question + zero tools catches *any* wording |
| Retry never fired on test turns | Douglas's sessions run **private** — the guard skipped exactly the turns under test. Retry now inherits the turn's privacy |
| Watcher stall-nudged a suppressed placeholder | "[The Tailor held this reply" is not her stall |
| `[[SHAPE: blocked]]` visible in chat | Gate retry *replaces* fullContent after the strip. Strip now follows every content change |
| "jmap-sniff" confabulation (turn 1321) | She claimed a file **her own directory listing disproved** → **tool-result contradiction detector**: file-existence claims vs the turn's own complete listings |

## Act III — Sermons and Turn Lanes

> "The tool that makes the garment — the Tailor — needs to make choices about which tools would be helpful and which laws would be useful given what is in this turn… we remake this turn for turn."

**Sermons.** Per turn, the Tailor asks the Book of Doug what law speaks to the human's message (`book_find_relevant`, top-3 ACTIVE entries) and weaves it into the garment: *ruled law, not style guidance — and if you must break one, that's a Prayer, not a decision.* The Book stops being a system she must remember to consult and becomes part of what she's wearing when she thinks.

**Turn Lanes.** Deterministic lanes (calendar / mail / pricing v1) route the tools *this turn* needs back through the exploring-loom strip — per turn, never a permanent surface to recite. On a deterministic **miss**, the AI tier answers "does a tool exist for this?" against the catalog. Every outcome lands in `tool_lane_log`, so AI-found families graduate into deterministic lanes:

> "Last time you said cat and we didn't know to give you vet."

**The announcement.** First probe after the lanes shipped: the tool was routed in — and she *still* answered "I have no path to it" from her thread-trained self-model. A silently present tool is one she doesn't believe she has. The Tailor now says it in the garment: *"routed into your surface FOR THIS TURN ONLY — live and callable RIGHT NOW."*

## The closing scene

Next turn, asked for the next five calendar events, she:

1. **Called** `capcom__calendar_list_events` — the tool that hadn't existed for her yesterday;
2. Read the raw response and recognized it as **the bug she'd been assigned to fix** — past-date recurring masters leaking through the JMAP `after` filter, no recurrence expansion — the hypothesis, confirmed with ground truth, plus a duplicate birthday she spotted unasked;
3. Gave evidence first, then asked the one legitimate question: *"Do you want me to write that fix now?"*

That is the whole arc in one turn: from *"I can't answer that"* to *"here is the evidence, and I found your bug in it."*

---

## The parts, and where they live

| Piece | What it does | Home |
|---|---|---|
| Auto-retry | Bad-news verdicts re-drive the turn on a fresh Tally Max route | `stream.ts` |
| Shape Sentinel | `[[SHAPE: …]]` declares every reply's type | `aiPayload.ts` prompt, `tir.ts` parser |
| HeadGate | Head-hold watermark; hold-shapes verified or suppressed pre-delivery | `tir.ts`, chained in `courier.ts` |
| Shape fraud | Declared `result` + no tools + completion claims = provable lie | `tir.ts` |
| Contradiction detector | File claims contradicted by the turn's own listings | `tir.ts` (stage 1b4) |
| `wt_search` | Structured search helper — pattern is data, never shell text | `worktreeTools.ts` |
| Sermon | Per-turn Book of Doug doctrine in the garment | `sermon.ts` |
| Turn Lanes | Per-turn tool routing + AI tier on miss + graduation log | `turnLanes.ts`, `tool_lane_log` |
| Lane block | The Tailor announces what it routed | `aiPayload.ts` |

**Tests:** 139 in the gate cluster plus sermon/lanes suites — all green. **Deploys:** fourteen across two days, all through `dev-deploy.sh`, all live.

## Open questions (for the Weaver)

- Short-circuit lane: pure helper turns that skip the LLM entirely — Tailor-voiced?
- Sentinel compliance by model: gpt-5/deepseek good, o3 ~1/3. Fail-open covers it; a per-model nudge could close it.
- The chaining guard still blocks `|` inside quoted regex. Conservatism beats a parser bug — until someone writes the quote-aware scanner with real tests.
- Jane's offer stands: the recurrence-expansion fix for the Focus calendar filter, now with ground truth.

---

*Title chosen because both ends of it are yours, Douglas: the complaint and the result. The middle was a genuinely good day at the loom.*