--- title: How the Infer Tools Work description: "Written 2026-09-25. Covers the two CASE inference workers: the batch activity inferral (caseinferral) and the downward framework decomposition (casedownwardinferral). Both live in the tc-case-inferral repo, share one…" created: 2026-09-25 updated: 2026-09-25 authors: ThinkingCap R&D topics: [Skills] status: published canonical: https://console.thinkingcap.com/rd/Skills/INFER-TOOLS-how-they-work summary: "The Infer Tools are two CASE inference workers that run from a shared infrastructure: batch activity inferral maps learning activities to skills, while downward framework decomposition breaks down competencies into actionable skills. Both workers use the same Docker image, Postgres queue system, and LLM routing layer called Tally, with AI reranking validated against real skill candidates to prevent hallucination." audio: https://thinkingcap.blob.core.windows.net/rd-home/summaries/605c15d78d73520803e20c091957bb88f27c7e090f791ef69e44be858a222fe3.mp3 audio_full: https://thinkingcap.blob.core.windows.net/rd-home/summaries/full-a36a502b5ff789ff3d8ad2f3689c4f2e40157f733cf71f38f132167c469ed5a4.mp3 --- # How the Infer Tools Work *Written 2026-09-25. Covers the two CASE inference workers: the batch activity inferral (`case_inferral`) and the downward framework decomposition (`case_downward_inferral`). Both live in the `tc-case-inferral` repo, share one Docker image (`caseinferralworker`), drain the same Postgres queue substrate, and are driven from the Skills surface.* --- ## 0. The two directions | | **CASE Inferral** (batch) | **CASE Downward Inferral** | |---|---|---| | Question answered | "Which skills is this **activity** about?" | "What does this **competency** break down into?" | | Input | A client's learning activities (`Course` rows) | A node in the org's OWN framework | | Output | Skill pins per activity per framework | Child nodes, overlay mappings, edges | | Queue type | `case_inferral` | `case_downward_inferral` | | Entry point | `dist/worker/index.js` | `dist/downward/index.js` | | Surface tab | Activities / Skills / Frameworks | Our Frameworks | They are mirrors: one maps content **up** into public taxonomies; the other decomposes your own competencies **down** toward actionable terminal skills (RSDs), with the public taxonomies riding along as *overlay mappings* — never merged into your graph. --- ## 1. Shared substrate: maestro_queues Both workers drink from the same table — `maestro_queues` in the fleet's `maestroqueues` Postgres database (v2 of patch_jobs). One row = one job. - **Lease, not lock.** A worker claims a row (`FOR UPDATE SKIP LOCKED`), sets `lease_expires_at`, and renews on a 60 s heartbeat (lease 120 s). Crash or hang and the row becomes claimable again — that *is* the retry. - **Poison guard.** `dequeue_count >= max_dequeues` = poison, never reclaimed. Rate-limit (429) failures *undo* the dequeue increment and back off 5 min, so provider capacity alone can never poison a job. - **Dedupe.** A partial unique index on `dedupe_key` over live (pending/in_progress) rows makes enqueues idempotent — re-clicking "Run discovery" while a job is live is a no-op. Scoped reassess jobs carry a scope suffix in the key so scoped and full jobs coexist. - **Any number of workers** can drain in parallel; claims are atomic. The LLM is reached through **Tally** (the routing/bandit layer): Tally recommends a model per call, the worker executes it via the matching adapter, and every call reports back telemetry + a deterministic quality score (fraction of picks that are real candidate-set members — hallucination/mis-key rate, no model call needed). The production pin is `kimi-k3` (~$0.039/call). --- ## 2. CASE Inferral (batch): activity → skills ### 2.1 Job creation `dist/jobCreation/index.js` (scheduled, or single-client via `JOB_CREATION_CLIENT`): 1. Enumerate tenants from the patch `clients` registry. 2. Per tenant, list **active** activities from the client's own MSSQL (`Course` where `IsActive=1 AND EditorStep=0`, retired types excluded). 3. Content-hash each (title+description+type+modified) — only activities new *or changed* at the current `analysis_version` are enqueued. Bumping the version forces re-analysis. ### 2.2 The two stages (the important part) **Stage 1 — deterministic retrieval** (`worker/retrieval.ts`, `case/catalog.ts`). For each *enabled* framework, the database finds candidate skills directly from the raw evidence (title, description, keywords, SCORM text, CMI text — no AI summarization): - keyword: Postgres FTS over the taxonomy (per-framework candidate node types only — containers never pollute candidates); - vector: pgvector cosine (gated, taxonomy frameworks only); - related: one-hop relationship expansion around the top keyword hits. Merged by a fixed weighted score with a stable tie-break. **Same input → same candidate set, 100% of the time** (measured Jaccard 1.000). Canonical frameworks (Red Seal, OaSIS) retrieve from `skill_framework_entities` the same way; Red Seal candidates are restricted to `task/sub_task/skill/concept`. **Stage 2 — AI rerank** (`worker/inferral/`). The model never searches; it only ranks the frozen candidates. Vote mode (the default, validated on ARCA): - **5 independent passes** over the *same* candidate set; - a skill must survive **≥4/5 votes** to commit; - median-confidence floor 0.40; - per-framework cap (3 skills), canonical names enforced, confidence clamped; - picks are attributed by **id membership**, not the JSON key the model used — a mis-keyed or invented id scores zero (the anti-hallucination guard that also feeds Tally's quality term). Cost: 5 model calls ≈ **$0.20/activity** (RECO: 1,327 activities ≈ $265). ### 2.3 Results & the human loop Rows land in `inferral_results` keyed (tenant, activity, version, framework, skill), status `Inferred`. Re-runs replace **only the frameworks that were evaluated** (scoped delete), so a scoped reassess never disturbs the rest. - **Pins** (assessment chat / manual) are written at reserved `analysis_version 0`, status `Pinned` — batch re-runs never touch them, and read paths always include them. - **Reject** (reviewer QA) marks the machine rows `Rejected`, requeues the activity at the same version, and posts a quality score to Tally keyed by the exact session id — the bandit steers away from models whose picks humans keep rejecting. ### 2.4 Which frameworks get evaluated — per organization The three base taxonomies (SCT, ESCO, O*NET) are always on. Canonical frameworks (Red Seal, OaSIS, future imports) are **opt-out**: every framework visible to the org defaults ON; the Frameworks tab's checklist (saved all at once) writes `skill_framework_selections`, and the worker resolves the enabled set per job (60 s cache). Turning a framework **on** fires *one* reassess pass: a job per active activity carrying *all* newly-enabled frameworks in `payload.frameworks` — the worker evaluates exactly that subset and leaves every other framework's rows untouched. Turning one **off** deletes its machine rows (pins survive). --- ## 3. CASE Downward Inferral: competency → decomposition ### 3.1 The data model (patch DB, schema/004) - `org_frameworks` — a branch-owned CASE document (authored, imported, or adopted from the registry); - `org_framework_nodes` — its tree (source `authored`/`imported`/`inferred`; review `confirmed`/`pending`/`superseded`); - `org_node_mappings` — overlays onto external frameworks (relationship type, confidence, resolved labels, `update_available` when a source re-imports); - `org_framework_edges` — typed cross-links; - `downward_runs` — every job's guards, stats, cost, state. ### 3.2 Jobs Enqueued from the Our Frameworks tab — **Infer Down** on a node, or **Run discovery** to sweep from the roots — or by script. Payload: tenant, branch, framework, node, mode (`infer_down`/`discovery`), analysis version. ### 3.3 The worker `dist/downward/index.js` — same lease machinery, same Tally-routed model. `inferDown`: 1. Reads the node + its framework context. 2. Decomposes it into children (deterministic guard rails in `downward_runs.guards` bound fan-out), grounding proposals in catalog candidates (the same `skillCatalog` the batch worker uses) and registry labels. 3. Writes children as `source=inferred, review_status=pending`, overlay mappings with confidence + reasoning, and typed edges. Alternative decompositions are kept as **divergence branches** (cluster labels on the CFItem payload). 4. Discovery mode enqueues the next level down, so a run grows the tree level by level toward RSDs (terminal states: `rsd`, `rsd_candidate`, `unresolved_terminal`). ### 3.4 Provisional truth + the kill switch Inferred nodes/mappings are **live immediately** (pending review) — humans are the kill switch, not a gate: - **Reject a mapping** → every inferred descendant grounded *solely* by it is superseded (its own mappings superseded too). - **Reject a node** → its whole inferred subtree is superseded. Nothing is ever deleted — superseded rows are provenance, hidden from the tree. Confirmed rows are human-curated from then on. ### 3.5 The surface Our Frameworks tab: cards per branch framework (node/inferred/pending/RSD counts, last run), the full tree (inferred nodes tinted + sparkle; "Inferred branches only" filter keeps ancestors as the path), node dialog with overlay badges, confirm/reject, add/edit child, infer-down. **Download JSON** exports the framework + all inferences as a CASE-shaped package (re-importable; provenance/mappings/edges in `TCExtensions`). Create flows: author / import CASE (upload or paste, server-side validation, downloadable template) / adopt a registry framework or subtree (seed mappings at confidence 1.0). --- ## 4. Where things stand (2026-09-25) **Live today:** batch worker code + per-org framework selection + Red Seal canonical retrieval (srv/main); RECO fully swept (1,327 activities, 5,118 skills); RECO's PLIP/Pan-Can org framework imported as a tree (224 nodes: 5 domains → 25 → 194); the downward worker registered as `service-case-downward-inferral` in the `skills` group beside CASE Inferral, with the cloud-init env block wired for the whole family (also fixing the batch worker's long-standing env gap). **Awaiting operator rolls (Douglas):** case-worker image rebuild, skills surface deploy, capcom web/api deploy, then worker-VM reimage so the new env block takes effect — after which `Run discovery` drains end-to-end. --- *Appendix — key tables:* `maestro_queues` (jobs), `inferral_results` (batch projection), `skill_taxonomy` + `skill_frameworks/_entities/_relationships` (reference frameworks), `skill_framework_selections` (per-org opt-outs), `org_frameworks/_nodes/_mappings/_edges` + `downward_runs` (org frameworks).