---
title: "Intent Engine — Sprint Plan"
description: "Companion to intent-engine-plan.md (updated 2026-09-15, post research-brief comparison). Six 1-week sprints, phases P0–P5 mapped 1:1 onto sprint goals; P6 stays a pair of decision points with entry criteria, not…"
created: 2026-09-22
updated: 2026-09-30
authors: ThinkingCap
topics: [Research Development]
status: published
canonical: https://console.thinkingcap.com/guest/Research-Development/CapCom/intent-engine-sprint-plan
date: 2026-09-22
---

# Intent Engine — Sprint Plan

Companion to `intent-engine-plan.md` (updated 2026-09-15, post research-brief comparison).
Six 1-week sprints, phases P0–P5 mapped 1:1 onto sprint goals; P6 stays a pair of
decision points with entry criteria, not scheduled work. Team: Campbell + Claude.
Cadence note: the P1 shadow soak is calendar time, not effort — it overlaps Sprint 4.

---

## Sprint 1 — "Prime the pump" (P0, part 1: data + pure routing machinery)

**Goal:** the eval corpus exists, built from saved Console chat, and the pure
(no-IO, no-wiring) routing pieces are built and unit-tested.

| # | Story | Est (d) |
|---|-------|---------|
| 1.1 | **Corpus extraction** — `scripts/extract-intent-corpus.ts`: pull `role='user' AND kind='typed'` turns from `app_chat_message` joined to `app_chat_session`; label each turn by its following assistant message — content contains a `surface:forms` marker → positive (parse `schema=<blobDir>` + fill/records from the marker URL); no marker → OOS negative. Skip canned pairs (`generator='surface-bridge'`), right-click kinds, attachment turns. Emit JSONL + a stats report (turn count, marker rate, top-20 forms by open frequency). | 1.5 |
| 1.2 | **Corpus hygiene pass** — strip/hash emails, names, client identifiers before anything lands in `tests/fixtures/`; the committed fixture must be safe to push. Stats report reviewed by Campbell (this is also where the top-20-forms list for paraphrase/negatives work comes from). | 0.5 |
| 1.3 | `src/lib/intent/routingTable.ts` — per-dir examples from `intentExamples` + templates over `title/entity/keywords/description`; `classifyVerb(text): 'fill'\|'records'`. Pure builder, DB-free. | 1 |
| 1.4 | `src/lib/intent/guards.ts` — `hasBearingSlots(text)` (email regex, quoted strings, `named\|called\|name is\|@`, `key=value`) + `hasNegationCue(text)` (`don't\|do not\|not yet\|hold off\|instead of\|rather than`). Tests for both, incl. tricky cases ("don't" inside a quoted value, "not" in a form title). | 1 |
| 1.5 | `src/lib/intent/embeddings.ts` — OpenAI `text-embedding-3-small` wrapper, 800ms timeout, ETag-keyed promise-locked class-vector cache aligned with `catalogCache`. | 1 |

**Exit:** extraction report reviewed; top-20 form list agreed; routingTable/guards/embeddings
unit tests green; `npm run typecheck` clean. Nothing wired into `chat.ts`.

---

## Sprint 2 — "Decision engine" (P0, part 2: decide() + harness + operating point)

**Goal:** `decideChatIntent()` exists and the harness proves an operating point at
precision ≥ 0.98 on a real eval set. **P0 accept.**

| # | Story | Est (d) |
|---|-------|---------|
| 2.1 | `src/lib/intent/router.ts` — `decideChatIntent({text, availableSurfaces, formsGranted})` + `scoreIntents(text)`; threshold + margin + cross-family ambiguity rule; grant gate re-verified at decision time; guard refusal (slots + negation) returns null. | 1.5 |
| 2.2 | `scripts/eval-intent-router.ts` — builds table from live catalog or fixture; per-class precision/recall; confusion pairs; (threshold × margin) sweep; `--check` exits non-zero below 0.98. | 1.5 |
| 2.3 | **Eval set assembly** — catalog `intentExamples` (positives) + hand paraphrases for the Sprint-1 top-20 forms + records phrasings + mined negatives (Sprint 1 corpus) + explicit negatives from the research brief: question-form ("how do I…", "what do I need to…", "where do I find…") per top-20 form, negated commands, adversarial (typos/slang/indirect/conditional). | 1.5 |
| 2.4 | `src/config.ts` intent env section (`INTENT_*` flags, `optional()` pattern) + network-free CI fixture `tests/fixtures/intent-embeddings.json.gz`; `tests/intentRoutingTable.test.ts`, `tests/intentDecision.test.ts`. | 0.5 |

**Exit:** harness report with chosen defaults (`INTENT_THRESHOLD`/`INTENT_MARGIN`) at
precision ≥ 0.98; confusion pairs reviewed with Campbell (this surfaces the
`intentExamples`-quality risk #2 — per-dir curation feed goes to the Forminator team);
`npx tsx scripts/eval-intent-router.ts --check` green; typecheck + `tsx --test` green.

---

## Sprint 3 — "Shadow" (P1: wire + soak)

**Goal:** decisions computed and logged on every eligible typed turn in prod, zero
behavior change. Soak starts. **P1 accept.**

| # | Story | Est (d) |
|---|-------|---------|
| 3.1 | `src/routes/chat.ts` — pre-`continueChat` block: eligibility guard (`kind==='typed'`, no attachments/choiceAnswer, `surfaceAware===true`, envs set) → `decideChatIntent` → `intent_route` event → always fall through (`INTENT_LOG_ONLY=true`). Response shapes byte-identical; test that proves it. **Amended 2026-09-22 (Campbell): canvas turns are logged with `canvas:true`, not vetoed** — the console eagerly adopts a canvas session per sign-in, so a veto starved the soak; the canvas veto moves to Sprint 4's acting path. | 1.5 |
| 3.2 | **Soak query pack** — saved queries for the week: outcome counts (`routed`/`fell_through`/`shadow`), embed p99, daily sampled review of would-have-routed turns vs what the model actually did (override/disagreement list), threshold-sensitivity replay from logged confidence+margin. | 1 |
| 3.3 | Commit → operator roll with `INTENT_LOG_ONLY=true`. (Claude commits + `az acr build`; console-api roll is Douglas/operator.) | 0.5 |

**Soak:** ≥ 1 week calendar, overlaps Sprint 4. Enable gate for Sprint 4's flip:
live-measured precision from the sampled review ≥ target — not the offline number.

---

## Sprint 4 — "Short-circuit" (P2: routed fill + records)

**Goal:** high-confidence form intents route with no LLM call; kill-switch reverts
next turn. **P2 accept.**

| # | Story | Est (d) |
|---|-------|---------|
| 4.0 | **Engine swap — LOCAL MiniLM becomes the default embedder** *(added + landed 2026-09-22, Campbell — the bake-off preceded Sprint 4 on day-1 soak evidence)*: `src/lib/intent/localEmbedder.ts` (q8 MiniLM, mean+norm, truncation guard, load-failure self-clearing); `embeddings.ts`/`router.ts` dispatch on `INTENT_EMBED_IMPL` (OpenAI hot rollback); config operating point follows the impl (0.80 local / 0.75 openai); committed fixture regenerated in MiniLM space (eval **98.5/78.9/44.7**, `--check` green); structural tests rewritten (16/16, suite 455/0); Dockerfile bakes the weights (+sharp-native idiom, preload = image self-test). Soak restarts on the local engine at roll. | 1 |
| 4.1 | Synthesized reply path — one prose line + exact marker shape (`chat.ts:936` fill / `:959` records); streaming parity (same headers, `writeControl({type:'done'})`, conditional payload keys absent like a no-canvas model turn); persist via `insertMessage(...,'intent-router:v1',...)`. | 2 |
| 4.2 | `tests/intentSynthesis.test.ts`; `docs/E2E-FORMCAST.md` +3 manual checks (routed fill, routed records, slot-bearing falls through) → 34 total. | 1 |
| 4.3 | **Mid-sprint go/no-go:** soak review with Campbell — sampled live precision, disagreement analysis, p99 latency. On go: operator flips `INTENT_ENABLED=true`; verify live (chat "configure the branch basic settings" → FormCast auto-opens, `intent-router:v1` in transcript, zero LLM calls in logs); verify kill-switch reverts next turn. | 1 |
| 4.4 | Soak backfill — if the review says "not yet," sprint slack goes to threshold tuning + eval-set fixes from live disagreements. | (buffer) |

**Exit:** E2E 34/34 (Campbell's browser checklist); routed turns segment in `chat_reply`
dashboards via `generator`.

---

## Sprint 5 — "One router" (P3 tool-path swap + P4 surface intents)

**Goal:** the embedding router serves both consumers; curated surface navigation joins
at the higher threshold. **P3 + P4 accepts.**

| # | Story | Est (d) |
|---|-------|---------|
| 5.1 | `catalog.ts` — `routeIntent` dispatch on `FORMS_ROUTER_IMPL=embed\|llm\|keyword`; candidate-mapping + keyword-fallback tests; measure haiku-classify removal from model-driven opens (before/after latency in the PR description). | 1.5 |
| 5.2 | `src/lib/intent/surfaceIntents.ts` — curated surface phrase table; second routing-table source; `INTENT_SURFACE_THRESHOLD=0.82`; ambiguity + ungranted-surface fall-through tests. | 1.5 |
| 5.3 | Eval additions for surface intents (positives, cross-family ties, ungranted targets); `--check` still ≥ 0.98 at the surface threshold. | 1 |
| 5.4 | Operator roll; env rollback drill (`FORMS_ROUTER_IMPL=llm`). | 0.5 |

---

## Sprint 6 — "Seeding" (P5: slot seeding, verifiable fields only)

**Goal:** routed opens can pre-seed enum/boolean fields; the intent object takes the
research brief's canonical shape. **P5 accept.** First sprint that may need a
console-app roll (chips) — everything before this is server-only.

| # | Story | Est (d) |
|---|-------|---------|
| 6.1 | `IntentAction` gains `entities: Record<string,string>` + `missing: string[]` (brief-canonical shape — decision object stays the single classifier↔synthesis contract). | 0.5 |
| 6.2 | Extraction restricted to enum/boolean matches via pure `choicePlan`; synthesized `surfaceActions` ride `deliverFormCastCommands` unchanged; free-text never seeded (guards still force fall-through). | 2 |
| 6.3 | Optional first-question chips via pure `computeChoices`. | 1 |
| 6.4 | Tests (seeded batch shape, no-seed-on-guard cases) + E2E additions; operator rolls (console-api + possibly console-app). | 1 |

---

## Not scheduled — P6 decision points (entry criteria, not dates)

- **6a Calibration** — enter when shadow/live data shows global threshold+margin is
  insufficient (per-class confusion persisting after eval-set fixes). First step is
  offline per-class thresholds (TunableThreshold, TS, JSON artifact). AutoIntent
  sidecar only if that also fails: pinned version, `Pipeline.load` offline artifact,
  behind the unchanged `decideChatIntent` seam, TS router hot as fallback.
- **6b Mutations** — enter only when routed turns start *writing* (submit /
  create / update / delete — not seeding a user-reviewed open). Then adopt the
  brief's registry (`permission`/`confirmation`/`risk` per action) + confirmation
  step + XState-style workflow machine. Rejected for v1–v5 on ops-burden grounds;
  the risk class changing is what reopens it.

## Deploy / roll gates (estate rules, restated)

- Claude commits (+ `az acr build`) only. Console-api rolls: Douglas/operator — needed at end of Sprints 3, 4, 5, 6.
- Console-app (client): untouched through Sprint 5; first possible client roll is Sprint 6 (chips).
- Every flip is an env change (`INTENT_ENABLED`, `INTENT_LOG_ONLY`, `FORMS_ROUTER_IMPL`) — rollback is an env flip + operator roll, never a code revert.

## Working agreements

- **Precision-first discipline:** no sprint's enable gate is skipped on the strength of the previous sprint's offline numbers. The 0.98 `--check` gates Sprint 3; the *live* sampled review gates Sprint 4's flip.
- Eval fixtures are committed but sanitized (Sprint 1.2) — no client PII in the repo.
- Risk watch: #2 (`intentExamples` quality) surfaces in Sprint 2's confusion report → Forminator curation backlog; #6 (negation/speech-act) is covered by Sprint 1.4 guards + Sprint 2.3 negatives, with Sprint 3's disagreement review as the live control.

## Timeline

6 weeks of work, ~6.5 calendar weeks with the soak overlap:

```
Wk1   Wk2   Wk3   Wk4   Wk5   Wk6   Wk7
S1    S2    S3    S4    S5    S6
            |-- soak --|
                  go/no-go (S4.3)
```


---

## Sprint 4.5 — "Correction sidecar" (soak instrumentation — added 2026-09-23, Campbell)

**Goal:** measure what an LLM correction layer *would* buy before anyone lets
it act. On correctable fall-throughs, a cheap model suggests the intended
phrasing ("ad a user" → "add a user"), logged on the same shadow event;
offline eval measures the accuracy. Nothing user-visible changes.

| # | Story | Est (d) |
|---|-------|---------|
| 4.5.1 | `src/lib/intent/correction.ts` — suggestCorrection: haiku/gpt-4.1-mini auto-pick (joshua-adjudicator policy, DeepSeek excluded); strict fail-closed verdict parse; AbortSignal timeout; never throws; failures booked as `verdict:"error"` rows. Config: `INTENT_CORRECTION_ENABLED` (default OFF), `_MODEL`, `_TIMEOUT_MS` | 1 |
| 4.5.2 | shadow.ts wiring — corrector runs on below-threshold/margin/cross-family-margin ONLY (guard-declined turns never reach it); `payload.correction` always-present key; `xyzzy` PII pin: derived text only, raw turn never in payload (single-text-surface amendment — Campbell sign-off) | 0.5 |
| 4.5.3 | Intent Logs surfacing — admin whitelist line + capcom row-dialog "Suggested correction" block | 0.5 |
| 4.5.4 | `scripts/eval-intent-correction.ts` + `tests/fixtures/intent-correction-eval.jsonl` — fixture arms (verdict accuracy, routing delta, oracle gap), `--project` full-set projection vs the bake-off baseline, `--live` distribution + label dump, `--labels` scoring | 1 |
| 4.5.5 | Offline eval report → Campbell sign-off on the derived-text amendment → operator flips `INTENT_CORRECTION_ENABLED=true` → 1–2 wk soak → labeled-sample scoring feeds any acting-path proposal | (buffer) |

**First numbers (2026-09-23, gpt-4.1-mini, offline):** fixture rewrite
precision 97.0% (the one false open = a value-bearing turn the production
guard-slots exclusion already blocks), 24/32 rewrites routed to the expected
form, **0 wrong routes**; full-set projection 98.5→98.0% precision /
78.9→81.8% recall / 44.7→46.6% coverage — buys recall at half a point of
precision, landing exactly on the 98.0 gate; ~$0.02/month at observed volume.
Any ACTING-path correction pre-pass is a separate go/no-go fed by these
numbers and the labeled soak sample — not part of Sprint 4's flip.