---
title: "Intent Engine — Sprint Plan"
description: "Companion to intent-engine-plan.md (updated 2026-09-15, post research-brief comparison). Six 1-week sprints, phases P0–P5 mapped 1:1 onto sprint goals; P6 stays a pair of decision points with entry criteria, not…"
created: 2026-09-21
updated: 2026-09-21
authors: ThinkingCap R&D
topics: [CapCom]
status: published
canonical: https://console.thinkingcap.com/rd/CapCom/intent-engine-sprint-plan
summary: The Intent Engine Sprint Plan outlines a six-sprint development schedule for building a routing system that classifies user chat intents to automatically open forms without LLM calls. The plan starts with data collection and core routing logic in Sprint 1, validates precision through offline evaluation in Sprint 2, then shadow-deploys with logging in Sprint 3 before enabling live routing in Sprint 4. Subsequent sprints consolidate routing implementations and add slot-seeding capabilities.
audio: https://thinkingcap.blob.core.windows.net/rd-home/summaries/7874725dff872042b4654f1284aa46fc0ba2d2490e4c86c2702997f803c509c7.mp3
date: 2026-09-21
---

# Intent Engine — Sprint Plan

Companion to `intent-engine-plan.md` (updated 2026-09-15, post research-brief comparison).
Six 1-week sprints, phases P0–P5 mapped 1:1 onto sprint goals; P6 stays a pair of
decision points with entry criteria, not scheduled work. Team: Campbell + Claude.
Cadence note: the P1 shadow soak is calendar time, not effort — it overlaps Sprint 4.

---

## Sprint 1 — "Prime the pump" (P0, part 1: data + pure routing machinery)

**Goal:** the eval corpus exists, built from saved Console chat, and the pure
(no-IO, no-wiring) routing pieces are built and unit-tested.

| # | Story | Est (d) |
|---|-------|---------|
| 1.1 | **Corpus extraction** — `scripts/extract-intent-corpus.ts`: pull `role='user' AND kind='typed'` turns from `app_chat_message` joined to `app_chat_session`; label each turn by its following assistant message — content contains a `surface:forms` marker → positive (parse `schema=<blobDir>` + fill/records from the marker URL); no marker → OOS negative. Skip canned pairs (`generator='surface-bridge'`), right-click kinds, attachment turns. Emit JSONL + a stats report (turn count, marker rate, top-20 forms by open frequency). | 1.5 |
| 1.2 | **Corpus hygiene pass** — strip/hash emails, names, client identifiers before anything lands in `tests/fixtures/`; the committed fixture must be safe to push. Stats report reviewed by Campbell (this is also where the top-20-forms list for paraphrase/negatives work comes from). | 0.5 |
| 1.3 | `src/lib/intent/routingTable.ts` — per-dir examples from `intentExamples` + templates over `title/entity/keywords/description`; `classifyVerb(text): 'fill'\|'records'`. Pure builder, DB-free. | 1 |
| 1.4 | `src/lib/intent/guards.ts` — `hasBearingSlots(text)` (email regex, quoted strings, `named\|called\|name is\|@`, `key=value`) + `hasNegationCue(text)` (`don't\|do not\|not yet\|hold off\|instead of\|rather than`). Tests for both, incl. tricky cases ("don't" inside a quoted value, "not" in a form title). | 1 |
| 1.5 | `src/lib/intent/embeddings.ts` — OpenAI `text-embedding-3-small` wrapper, 800ms timeout, ETag-keyed promise-locked class-vector cache aligned with `catalogCache`. | 1 |

**Exit:** extraction report reviewed; top-20 form list agreed; routingTable/guards/embeddings
unit tests green; `npm run typecheck` clean. Nothing wired into `chat.ts`.

---

## Sprint 2 — "Decision engine" (P0, part 2: decide() + harness + operating point)

**Goal:** `decideChatIntent()` exists and the harness proves an operating point at
precision ≥ 0.98 on a real eval set. **P0 accept.**

| # | Story | Est (d) |
|---|-------|---------|
| 2.1 | `src/lib/intent/router.ts` — `decideChatIntent({text, availableSurfaces, formsGranted})` + `scoreIntents(text)`; threshold + margin + cross-family ambiguity rule; grant gate re-verified at decision time; guard refusal (slots + negation) returns null. | 1.5 |
| 2.2 | `scripts/eval-intent-router.ts` — builds table from live catalog or fixture; per-class precision/recall; confusion pairs; (threshold × margin) sweep; `--check` exits non-zero below 0.98. | 1.5 |
| 2.3 | **Eval set assembly** — catalog `intentExamples` (positives) + hand paraphrases for the Sprint-1 top-20 forms + records phrasings + mined negatives (Sprint 1 corpus) + explicit negatives from the research brief: question-form ("how do I…", "what do I need to…", "where do I find…") per top-20 form, negated commands, adversarial (typos/slang/indirect/conditional). | 1.5 |
| 2.4 | `src/config.ts` intent env section (`INTENT_*` flags, `optional()` pattern) + network-free CI fixture `tests/fixtures/intent-embeddings.json.gz`; `tests/intentRoutingTable.test.ts`, `tests/intentDecision.test.ts`. | 0.5 |

**Exit:** harness report with chosen defaults (`INTENT_THRESHOLD`/`INTENT_MARGIN`) at
precision ≥ 0.98; confusion pairs reviewed with Campbell (this surfaces the
`intentExamples`-quality risk #2 — per-dir curation feed goes to the Forminator team);
`npx tsx scripts/eval-intent-router.ts --check` green; typecheck + `tsx --test` green.

---

## Sprint 3 — "Shadow" (P1: wire + soak)

**Goal:** decisions computed and logged on every eligible typed turn in prod, zero
behavior change. Soak starts. **P1 accept.**

| # | Story | Est (d) |
|---|-------|---------|
| 3.1 | `src/routes/chat.ts` — ~25-line pre-`continueChat` block (~:566): eligibility guard (`kind==='typed'`, no attachments/choiceAnswer/canvas, `surfaceAware===true`, envs set) → `decideChatIntent` → `intent_route` event → always fall through (`INTENT_LOG_ONLY=true`). Response shapes byte-identical; test that proves it. | 1.5 |
| 3.2 | **Soak query pack** — saved queries for the week: outcome counts (`routed`/`fell_through`/`shadow`), embed p99, daily sampled review of would-have-routed turns vs what the model actually did (override/disagreement list), threshold-sensitivity replay from logged confidence+margin. | 1 |
| 3.3 | Commit → operator roll with `INTENT_LOG_ONLY=true`. (Claude commits + `az acr build`; console-api roll is Douglas/operator.) | 0.5 |

**Soak:** ≥ 1 week calendar, overlaps Sprint 4. Enable gate for Sprint 4's flip:
live-measured precision from the sampled review ≥ target — not the offline number.

---

## Sprint 4 — "Short-circuit" (P2: routed fill + records)

**Goal:** high-confidence form intents route with no LLM call; kill-switch reverts
next turn. **P2 accept.**

| # | Story | Est (d) |
|---|-------|---------|
| 4.1 | Synthesized reply path — one prose line + exact marker shape (`chat.ts:936` fill / `:959` records); streaming parity (same headers, `writeControl({type:'done'})`, conditional payload keys absent like a no-canvas model turn); persist via `insertMessage(...,'intent-router:v1',...)`. | 2 |
| 4.2 | `tests/intentSynthesis.test.ts`; `docs/E2E-FORMCAST.md` +3 manual checks (routed fill, routed records, slot-bearing falls through) → 34 total. | 1 |
| 4.3 | **Mid-sprint go/no-go:** soak review with Campbell — sampled live precision, disagreement analysis, p99 latency. On go: operator flips `INTENT_ENABLED=true`; verify live (chat "configure the branch basic settings" → FormCast auto-opens, `intent-router:v1` in transcript, zero LLM calls in logs); verify kill-switch reverts next turn. | 1 |
| 4.4 | Soak backfill — if the review says "not yet," sprint slack goes to threshold tuning + eval-set fixes from live disagreements. | (buffer) |

**Exit:** E2E 34/34 (Campbell's browser checklist); routed turns segment in `chat_reply`
dashboards via `generator`.

---

## Sprint 5 — "One router" (P3 tool-path swap + P4 surface intents)

**Goal:** the embedding router serves both consumers; curated surface navigation joins
at the higher threshold. **P3 + P4 accepts.**

| # | Story | Est (d) |
|---|-------|---------|
| 5.1 | `catalog.ts` — `routeIntent` dispatch on `FORMS_ROUTER_IMPL=embed\|llm\|keyword`; candidate-mapping + keyword-fallback tests; measure haiku-classify removal from model-driven opens (before/after latency in the PR description). | 1.5 |
| 5.2 | `src/lib/intent/surfaceIntents.ts` — curated surface phrase table; second routing-table source; `INTENT_SURFACE_THRESHOLD=0.82`; ambiguity + ungranted-surface fall-through tests. | 1.5 |
| 5.3 | Eval additions for surface intents (positives, cross-family ties, ungranted targets); `--check` still ≥ 0.98 at the surface threshold. | 1 |
| 5.4 | Operator roll; env rollback drill (`FORMS_ROUTER_IMPL=llm`). | 0.5 |

---

## Sprint 6 — "Seeding" (P5: slot seeding, verifiable fields only)

**Goal:** routed opens can pre-seed enum/boolean fields; the intent object takes the
research brief's canonical shape. **P5 accept.** First sprint that may need a
console-app roll (chips) — everything before this is server-only.

| # | Story | Est (d) |
|---|-------|---------|
| 6.1 | `IntentAction` gains `entities: Record<string,string>` + `missing: string[]` (brief-canonical shape — decision object stays the single classifier↔synthesis contract). | 0.5 |
| 6.2 | Extraction restricted to enum/boolean matches via pure `choicePlan`; synthesized `surfaceActions` ride `deliverFormCastCommands` unchanged; free-text never seeded (guards still force fall-through). | 2 |
| 6.3 | Optional first-question chips via pure `computeChoices`. | 1 |
| 6.4 | Tests (seeded batch shape, no-seed-on-guard cases) + E2E additions; operator rolls (console-api + possibly console-app). | 1 |

---

## Not scheduled — P6 decision points (entry criteria, not dates)

- **6a Calibration** — enter when shadow/live data shows global threshold+margin is
  insufficient (per-class confusion persisting after eval-set fixes). First step is
  offline per-class thresholds (TunableThreshold, TS, JSON artifact). AutoIntent
  sidecar only if that also fails: pinned version, `Pipeline.load` offline artifact,
  behind the unchanged `decideChatIntent` seam, TS router hot as fallback.
- **6b Mutations** — enter only when routed turns start *writing* (submit /
  create / update / delete — not seeding a user-reviewed open). Then adopt the
  brief's registry (`permission`/`confirmation`/`risk` per action) + confirmation
  step + XState-style workflow machine. Rejected for v1–v5 on ops-burden grounds;
  the risk class changing is what reopens it.

## Deploy / roll gates (estate rules, restated)

- Claude commits (+ `az acr build`) only. Console-api rolls: Douglas/operator — needed at end of Sprints 3, 4, 5, 6.
- Console-app (client): untouched through Sprint 5; first possible client roll is Sprint 6 (chips).
- Every flip is an env change (`INTENT_ENABLED`, `INTENT_LOG_ONLY`, `FORMS_ROUTER_IMPL`) — rollback is an env flip + operator roll, never a code revert.

## Working agreements

- **Precision-first discipline:** no sprint's enable gate is skipped on the strength of the previous sprint's offline numbers. The 0.98 `--check` gates Sprint 3; the *live* sampled review gates Sprint 4's flip.
- Eval fixtures are committed but sanitized (Sprint 1.2) — no client PII in the repo.
- Risk watch: #2 (`intentExamples` quality) surfaces in Sprint 2's confusion report → Forminator curation backlog; #6 (negation/speech-act) is covered by Sprint 1.4 guards + Sprint 2.3 negatives, with Sprint 3's disagreement review as the live control.

## Timeline

6 weeks of work, ~6.5 calendar weeks with the soak overlap:

```
Wk1   Wk2   Wk3   Wk4   Wk5   Wk6   Wk7
S1    S2    S3    S4    S5    S6
            |-- soak --|
                  go/no-go (S4.3)
```