---
title: Intent Engine for the Lab Console Chat
description: "Campbell wants an intent engine between the console chat and FormCast: classify a chat turn, and on high confidence route directly to an action (open a form, show records, navigate a surface) with no LLM call — falling…"
created: 2026-09-21
updated: 2026-09-21
authors: ThinkingCap R&D
topics: [CapCom]
status: published
canonical: https://console.thinkingcap.com/rd/CapCom/intent-engine-plan
summary: Campbell's team is building a deterministic intent router that classifies chat requests and routes high-confidence ones directly to actions like opening forms, bypassing the language model to cut latency to 300ms. It uses embedding-based classification with threshold rules, falling back to the LLM when uncertain.
audio: https://thinkingcap.blob.core.windows.net/rd-home/summaries/03e79e8b10ab19afa0f71149d5199c8cea0400e05af6b255149001d1bae67166.mp3
date: 2026-09-21
---

# Intent Engine for the Lab Console Chat

## Context

Campbell wants an **intent engine** between the console chat and FormCast: classify a chat turn, and on high confidence route directly to an action (open a form, show records, navigate a surface) **with no LLM call** — falling through to the LLM when unsure. Precision-first: a wrong auto-route is worse than an LLM call.

**Why now:** the 2026-09-14 chat-deflection postmortem — the model never called `query_form_catalog` on "basic settings" even though `routeIntent` scored that route 0.95. Deterministic pre-LLM routing removes that failure class, cuts a form-open turn from ≥3 LLM calls to zero, and drops latency from multi-second to ~300ms.

**Confirmed decision (Campbell):** TS-native embedding router inside phoenix for v1; AutoIntent (DeepPavlov) stays the Phase-6 graduation path, adopted only if shadow-mode data shows threshold/margin calibration insufficient. We adopt AutoIntent's *decision contract* (threshold + margin + OOS → `null`) from day one.

**Companion document:** *Natural-Language Action Routing for Agentic Web Applications* (research brief, Campbell, 2026-09-15) — the general version of this architecture (canonical intent object → deterministic policy/authz/confirmation → execution). Compared 2026-09-15: this plan already implements its core principle (probabilistic language in, deterministic action boundary) for the lowest-risk action class; the catalog + `availableSurfaces` gate **is** its "action registry" pattern. Adopted from the brief: negation/speech-act/adversarial eval cases (P0) and the canonical `entities`/`missing` object shape (P5 seam). Deferred to P6b: its mutation-side layers (action-registry metadata, confirmation step, workflow machine).

## Key findings the plan builds on

- **Phoenix choke point:** `POST /chat` (`~/phoenix/src/routes/chat.ts:353`); all per-turn context (auth, persistence, telemetry, grants via `grantedSurfaceIds`→`availableSurfaces` :507-513, `formsGranted` :523, reconcile) is assembled **before** `continueChat(...)` (`:658`). Nothing upstream depends on model output — a short-circuit reuses the entire post-reply block (`:689-741`) and streaming framing unchanged.
- **Form-open needs no structured field:** assistant-text marker `[View <title>|surface:forms?view=fill&schema=<dir>]` (`src/lib/chat.ts:936`); client auto-fires it (tc-console `Chat.tsx:484-506` → `useSurfaces.launchSurface` → `formsCastRoutingRef.open`, App.tsx:304). **Zero client changes for v1.**
- **Catalog is the routing table:** `loadCatalog()` caches `schema-catalog.json` (ETag+5min TTL, `src/lib/forms/catalog.ts:69-93`) and doubles as the allow-list. `intentExamples` is declared (`catalog.ts:41`) but used **nowhere** — free training data.
- **No-model-turn precedent:** `POST /chat/surface-request` (`routes/chat.ts:763-786`) persists a canned pair with `generator='surface-bridge'`. Our routed turns mirror it with `generator='intent-router:v1'`.
- **Today's router is 2 tool-calls deep + spends a haiku classify:** `routeIntent` (`catalog.ts:163-202`), called only from the `query_form_catalog` tool executor (`src/lib/chat.ts:1738`). Deterministic `keywordRoute` (`catalog.ts:128-157`) stays as the offline floor (capped 0.4 — below short-circuit threshold by design).
- **Prefill reality:** client fills via `set_fields` surfaceAction batches (`FormCastView.applyCommandBatch`); no seed-on-open path exists.
- **AutoIntent facts:** v0.4.0, Apache-2.0, ~1 maintainer, torch+faiss CPU sidecar (1-2GB), shipped FastAPI server, OOS→null built in. Right-sized as a later calibration engine, wrong-sized as a v1 dependency for a ~50-class problem.

## Architecture (decided)

1. **Placement — server-side, pre-`continueChat` short-circuit** in `src/routes/chat.ts` (insert ~line 566, after reconcile; before stream headers ~640). Gates on the turn's own `availableSurfaces`, so grants/allow-list hold by construction. Client-side and hybrid placements rejected (client has no catalog/grants/persistence; splits the routing brain across two operator-rolled deploys).
2. **Classifier — embedding KNN over OpenAI `text-embedding-3-small`.** `openai@^6.38.0` is already a phoenix dep. Class vectors (few hundred 1536-dim) computed once per catalog ETag, held in memory; per-turn = one ~20-token embed (~100-200ms) + in-process cosine KNN (<1ms). Embed error/timeout (800ms) → fall through. Local ONNX and AutoIntent sidecar rejected for v1 (image weight / ops burden; revisit at Phase 6 with data).
3. **Intent model — two-stage:** classify to form identity (one class per catalog blobDir, ~50 classes), then a deterministic verb classifier resolves fill vs records. Non-form navigation intents come from a small curated surface table (Phase 4), never from the catalog.
4. **Fall-through guards — v1 extracts nothing, and guarded messages fall through.** Slot detectors (email regex, quoted strings, `named|called|name is|@`, `key=value`) → LLM, so typed values are never silently dropped. **Negation cues** (`don't|do not|not yet|hold off|instead of|rather than`) → LLM, because embedding cosine is weak on negation ("don't open the settings form" embeds ≈ "open the settings form") — the router's biggest known false-positive class. Only value-free, unnegated open-phrases short-circuit. Slot *seeding* (enum/boolean only, via pure `choicePlan`) is Phase 5.
5. **One router, two consumers:** pre-LLM path uses `decide()` (thresholded → one action or null); `query_form_catalog` keeps its `RouteCandidate[]` contract but swaps internals via `FORMS_ROUTER_IMPL=embed|llm|keyword` (default `embed`) — removes the per-turn haiku classify from model-driven opens too (Phase 3).

## Files

**New (~/phoenix):**
- `src/lib/intent/router.ts` — `decideChatIntent({text, availableSurfaces, formsGranted}) → IntentDecision | null`; `scoreIntents(text) → ScoredIntent[]` (shared with tool path). Threshold+margin+ambiguity rule, grant gate, guard refusal (slots + negation).
- `src/lib/intent/routingTable.ts` — pure builder: per-dir examples from `intentExamples` + templates over `title/entity/keywords/description`; `classifyVerb(text): 'fill'|'records'`.
- `src/lib/intent/embeddings.ts` — embeddings wrapper, 800ms timeout, ETag-keyed promise-locked vector cache aligned with `catalogCache`.
- `src/lib/intent/guards.ts` — fall-through detectors: `hasBearingSlots(text)` + `hasNegationCue(text)`.
- `src/lib/intent/surfaceIntents.ts` — curated surface phrase table (Phase 4).
- `scripts/eval-intent-router.ts` — eval harness (committed; posture of `/tmp/route-repro.ts`).
- `tests/fixtures/intent-eval.jsonl` + `tests/fixtures/intent-embeddings.json.gz` (network-free CI).
- `tests/intentRoutingTable.test.ts`, `tests/intentDecision.test.ts`, `tests/intentSynthesis.test.ts` (tsx --test, DB-free, `tests/testEnv.ts` pattern).

**Modified (~/phoenix):**
- `src/routes/chat.ts` — ~25-line short-circuit block + `intent_route` telemetry.
- `src/lib/forms/catalog.ts` — `routeIntent` dispatch on `FORMS_ROUTER_IMPL`; `loadCatalog`/`keywordRoute`/`compactIndex` untouched.
- `src/config.ts` — `intent` env section (same `optional()` pattern as `schemas`, :249-256).
- `docs/E2E-FORMCAST.md` — +3 manual checks (routed fill, routed records, slot-bearing falls through).

**Modified (~/tc-console): nothing.** Markers ride the existing wire shapes.

## Response synthesis (short-circuit turn)

- `reply.text` = one prose line + exact marker shape the system prompt teaches (`chat.ts:936` fill / `:959` records): `Opening the "<title>" form — say the details and I'll enter them, or type straight into it.\n\n[View <title>|surface:forms?view=fill&schema=<blobDir>]`. Records: `Here are your stored "<title>" records.` + records marker.
- Persist via the same `insertMessage(...,'intent-router:v1',...)` call site; existing `chat_reply` telemetry records `generator` → routed turns segment in every dashboard for free.
- Streaming parity: same headers, text as chunk(s), `writeControl({type:'done', ...payload})`, `res.end()`; conditional payload keys absent exactly like a no-canvas/no-choices model turn.
- **Eligibility guard:** only `kind==='typed'`, no attachments, no `choiceAnswer`, no canvas payload, `surfaceAware===true`, and `INTENT_ENABLED`/`INTENT_LOG_ONLY` set. Right-click turns never route.
- Continuity: next turn's reconcile + `formsProgressText` resumes the normal collection loop as if the model had opened the form.

## Intent schema

```ts
type IntentAction =
  | { kind: 'open-form-fill';    blobDir: string; title: string }
  | { kind: 'open-form-records'; blobDir: string; title: string }
  | { kind: 'open-surface';      surfaceId: string; label: string;
      target?: { view?: string; params?: Record<string,string> } };

interface IntentDecision {
  action: IntentAction; confidence: number; margin: number;
  matchedClass: string; matchedExample: string; embedMs: number;
}
```

`blobDir` ∈ `loadCatalog()` and `surfaceId` ∈ turn's `availableSurfaces` re-verified at decision time (mirrors the tool's registration+dispatch double-gate, `chat.ts:1740-1745`).

**P5 extension seam:** slot seeding extends `IntentAction` with the research brief's canonical `entities`/`missing` shape (`entities: Record<string,string>`, `missing: string[]`) rather than attaching `set_fields` directly to the decision — the decision object stays the single contract between classifier and synthesis.

## Eval, thresholds, rollout measurement

- **Harness** `scripts/eval-intent-router.ts`: builds table from live catalog or fixture, runs the eval set, prints per-class precision/recall, confusion pairs, (threshold × margin) sweep; `--check` exits non-zero below **precision ≥ 0.98**.
- **Eval set:** catalog `intentExamples` (positives) + hand paraphrases for top ~20 forms + records phrasings + **mined negatives** from `chat_send` history (turns whose replies had no forms marker — the OOS set that must fall through) + **explicit speech-act/robustness negatives** (adopted from the research brief):
  - *Question-form* phrasings for the top ~20 forms — "how do I…", "what do I need to…", "where do I find…" — must fall through (or route to help). The router's speech-act handling is otherwise implicit; this is what actually tests it.
  - *Negated commands* — "don't open the settings form", "actually don't show the records" — must fall through via `hasNegationCue`.
  - *Adversarial phrasings* — typos, slang, indirect wording ("I need a new X for…"), conditionals ("create it but don't save it").
  Negative quality is what makes precision-first real.
- **Thresholds (env-tunable, defaults set from the harness report, not intuition):** `INTENT_THRESHOLD=0.78`, `INTENT_MARGIN=0.05`, `INTENT_SURFACE_THRESHOLD=0.82`. Extra fall-through rules: slot + negation detection; cross-family ambiguity; embed error/timeout.
- **Shadow mode** `INTENT_LOG_ONLY=true`: compute + log every decision, never act. ≥1 week prod soak; enable is gated on **live-measured** precision vs what the model actually did — not the offline number.

## Flags (env, operator-flippable, no client deploy)

| Env | Default | Effect |
|---|---|---|
| `INTENT_ENABLED` | `false` | Master kill-switch. |
| `INTENT_LOG_ONLY` | `false` | Shadow: decide + log, always fall through. |
| `INTENT_THRESHOLD` / `INTENT_MARGIN` / `INTENT_SURFACE_THRESHOLD` | 0.78 / 0.05 / 0.82 | Tuning without redeploy. |
| `INTENT_EMBED_MODEL` | `text-embedding-3-small` | Embedding model. |
| `FORMS_ROUTER_IMPL` | `embed` | Tool-path internals: `embed`/`llm`/`keyword`. |

## Telemetry

One new server event (`recordServerEvent`, pattern at `routes/chat.ts:483`):
`intent_route: { outcome: 'routed'|'fell_through'|'shadow', kind, matchedClass, confidence, margin, runnerUpClass, embedMs, threshold, marginRequired, reason? }` — logged on every eligible typed turn when enabled or shadowing. Fell-through rows = recall data; routed rows join to `chat_reply` for override analysis (correction next turn, form dismissed in next surfaceContext sweep).

## Phases

- **P0 — Engine + harness, nothing wired.** All `src/lib/intent/*`, eval script, fixtures, P0 tests, config envs. *Accept:* harness report with operating point at precision ≥0.98; typecheck + tsx --test green.
- **P1 — Shadow mode.** Route guard + `decideChatIntent` + `intent_route` event; always falls through. *Accept:* zero behavior change (response shapes byte-identical), events flowing, ≥1wk soak, live precision ≥ target.
- **P2 — Short-circuit: form fill + records.** Synthesized reply path, `tests/intentSynthesis.test.ts`, E2E +3 checks (34 total). *Accept:* `INTENT_ENABLED=true` → routed turn persists pair, `chat_reply` carries `intent-router:v1`, conformant `done`, FormCast auto-opens with no client change; kill-switch reverts next turn; E2E 34/34.
- **P3 — Tool-path swap.** `catalog.ts` dispatch on `FORMS_ROUTER_IMPL`; candidate-mapping + keyword-fallback tests. *Accept:* haiku classify gone from model-driven opens (latency measured), candidate quality ≥ llm on eval set, env rollback.
- **P4 — Surface navigation intents.** `surfaceIntents.ts`, second routing-table source, eval additions. *Accept:* routes only at the higher surface threshold, only to granted surfaces, ambiguity falls through.
- **P5 — Slot seeding (verifiable fields only).** `IntentAction` gains the canonical `entities`/`missing` fields (research-brief shape); `set_fields` on routed turns restricted to enum/boolean matches via `choicePlan`; synthesized `surfaceActions` reuse `deliverFormCastCommands` unchanged; optional first-question chips via pure `computeChoices`.
- **P6 — Graduation (two independent decision points).**
  - **6a — Calibration.** Fit per-class thresholds offline from accumulated logs (TunableThreshold, in TS, JSON artifact). Only if data proves threshold/margin insufficient: stand up the AutoIntent sidecar — pinned version, `Pipeline.load` offline artifact, behind the unchanged `decideChatIntent` seam, TS router hot as fallback.
  - **6b — Mutations.** If routed turns ever start *writing* (submitting forms, create/update/delete actions — not just seeding an open, user-reviewed form), adopt the research brief's mutation-side layers: action registry with per-action `permission`/`confirmation`/`risk` metadata, an explicit confirmation step for mutating actions, and a deterministic workflow (XState-style classify → clarify → authorize → confirm → execute). Authorization stays server-side and structural exactly as today. These layers are rejected for v1–v5 for the same ops-burden reason as the AutoIntent sidecar; a mutation risk class is what changes that calculus.

## Risks

1. Embed-API latency/outage on typed turns — bounded by 800ms timeout + fall-through; shadow quantifies p99.
2. `intentExamples` quality unproven (dead data; sibling settings dirs may collide) — margin rule + confusion report surface it; per-dir curation is real content work (Forminator team).
3. Offline operating point optimistic vs live — Phase-1 soak is the control.
4. UX delta: routed open turn asks no first-field question — accepted; P5 closes it via `choicePlan`.
5. Multi-intent messages ("open X and show Y") — cross-family ties fall through; residual single-half routing is rare + low-harm.
6. **Negation/speech-act false positives** — embeddings score "don't open X" ≈ "open X", and "how do I X" ≈ "open X". Mitigated by `hasNegationCue` + explicit question-form/negated eval negatives; shadow-mode override analysis is the live control.
7. Grant-scope drift — structurally gated on the turn's `availableSurfaces`; mirror any future change (e.g. staff bypass) in tests.
8. AutoIntent + mutation-framework risk deferred, not eliminated — if P6a triggers: pin version, vendored artifact, TS fallback stays hot. P6b pulls in the brief's registry/confirmation/workflow layers only when the action risk class changes.

## Verification

- `npm run typecheck` + `npx tsx --test tests/*.test.ts` green in ~/phoenix (each phase).
- `npx tsx scripts/eval-intent-router.ts --check` at precision ≥0.98 before P1 enable.
- P2 live check (post operator roll): `INTENT_ENABLED=true`, chat "configure the branch basic settings" → FormCast opens on Home canvas, transcript shows `intent-router:v1` generator, no LLM calls in logs; flip env off → model behavior next turn.
- E2E-FORMCAST.md 34/34 (Campbell's browser checklist).
- Deploy rule (estate): Claude commits + `az acr build` only; console-api roll is Douglas/operator. Client untouched — no console-app roll needed until P5.