---
title: How I Think About Louie
description: Why Louie is not another queue, but the contract between surfaces, jobs, workers and data.
created: 2026-10-01
updated: 2026-10-01
authors: ThinkingCap
topics: [Research Development]
status: published
canonical: https://console.thinkingcap.com/guest/Research-Development/CapGPT/how-i-think-about-louie
cover: https://thinkingcap.blob.core.windows.net/guest-home/louie-pg-queues.png
summary: Louie is a unified abstraction layer that separates jobs from queue implementations, allowing browsers, backends, workers, and databases to communicate through a clean contract. It enables gradual migration of legacy systems without rewrites while improving security and observability.
audio: https://thinkingcap.blob.core.windows.net/guest-home/summaries/8b6dc890b6c265e637688b307007bf412b0d7fc5314cd06066a0cbee1b7c363d.mp3
audio_full: https://thinkingcap.blob.core.windows.net/guest-home/summaries/full-f76900458bc917bcf3f5947310437cde25f9cfed78002a355ab18f13da2d8f4c.mp3
---

# How I Think About Louie

![Postgres queue estate and Louie architecture](./louie-pg-queues.png)

I like this architecture a lot.

The core separation is clean:

**Browser knows surface. BFF knows identity. Louie knows jobs. Worker knows data. The database only sees RO workers.**

That is a very defensible data plane.

The important thing here is that **Louie isn't another queue**. Louie is the abstraction over all of our queues and workers.

Today we have a bunch of different implementations. Enrollment has its queue tables, details, failures and worker logs. SCORM has another implementation. Maestro has its unified queue and blobs. Then we have a dozen or so xAPI/settings families that are all basically doing variations of the same thing.

Underneath, they're all saying the same thing:

> Something needs to be done. Somebody needs to claim it, do the work, and either complete it or fail it.

Louie gives us one contract for that.

## Jobs and queues are different things

This is probably the most important distinction in the design.

`louie.jobs` is the ledger. It represents the fact that somebody asked us to do something.

For example:

> Generate this transcript for this user with this payload.

That job exists independently of how we happen to execute it.

`louie.queue_entries` is transport. It says:

> This job is waiting on this queue, it is available now, nobody has claimed it yet, and it has three attempts remaining.

So **a job is a business fact. A queue entry is an implementation detail.**

That separation feels right to me.

## Louie also knows what jobs are

`louie.job_types` is essentially Louie's registry.

It knows that `transcript.generate` exists, which queue handles it, how many attempts it gets, how long its lease lasts, whether it is enabled, etc.

Then `louie.job_type_schemas` defines the request and result contracts.

That means we aren't just throwing arbitrary JSONB around and hoping everybody agrees about what it means. A job type has a defined shape, and Louie can validate it.

## The security boundary gets much cleaner

The browser should know the surface it is talking to.

The BFF should know who the user is, what they are allowed to do, and how to turn that request into an authenticated envelope.

After that, the BFF should not need to know where queues live.

It should be able to say:

> Louie, here is an authenticated request for job type Y with this payload.

Louie decides where that work belongs, creates the job, enqueues it, and gives back a receipt.

Workers get an equally small contract:

> Claim. Complete. Fail.

They do not need broad table access either.

So instead of securing dozens of slightly different paths into the data plane, **Louie becomes the job boundary and the worker becomes the data boundary.**

The clean version of the path is:

```text
Browser
   ↓
Surface BFF
   ↓
Louie
   ↓
Queue
   ↓
Surface Worker
   ↓
Client DB
```

That distinction matters.

If every BFF writes `MaestroQueues` directly, then Louie is not really the single maître d'. The BFFs still know where the kitchen ticket rail is.

If Louie's purpose is truly that nothing upstream knows the queues or databases, then **Louie should own enqueueing, queue selection, receipts and probably result retrieval.**

The BFF owns HTTP, SSE and user authorization.

Louie owns job transport.

Workers own execution and database access.

Then the architecture really does become:

**The front end knows an API. The BFF knows identity. Louie knows jobs. Workers know databases. Nobody knows more than they need to.**

## We also finally get a proper history

`louie.job_events` gives us an actual lifecycle:

> submitted → queued → claimed → started → completed

Or whatever happened instead.

That means when something goes wrong, we can ask Louie what happened to job 123 rather than reconstructing the story from three tables and a pile of logs.

## We don't have to boil the ocean

This may be my favourite part of the architecture.

We do not have to rewrite Enrollment, SCORM, Maestro and every existing worker to get Louie into production.

Initially we can use **adapted mode**:

```text
caller → Louie → existing queue → existing worker
```

Louie records the canonical job and writes into the legacy queue. The existing worker does not even need to know Louie exists.

Then, workload by workload, we can move to **native mode**:

```text
caller → Louie → Louie queue → worker
```

At that point the old queue implementation can disappear.

So we can put the abstraction in place first and migrate the machinery underneath it over time.

## A few things I would tighten

### 1. Interactive latency

The 1 to 2 second normal latency bothers me more than almost anything else.

We have intentionally bought isolation at the cost of interactive latency, but most of that cost appears to be workers sleeping.

If execution itself is commonly 80 to 600 ms while queue wait is 500 to 1400 ms, I would attack the waiting, not the architecture.

`LISTEN/NOTIFY` could wake idle workers immediately, with polling still there as the recovery mechanism.

That should let us keep the same semantics while making the UI feel nearly instantaneous.

### 2. Dedupe should be based on authorization, not just scope

There is a subtle problem if the dedupe identity is based on things like:

```text
client + scopeKey + staff flag
```

but not the actor or the actual authorization context.

That is safe only while two people with identical scope are guaranteed to receive identical answers.

Eventually we will have operations involving things like:

- "me"
- preferences
- delegated permissions
- feature gates
- capability differences
- audit visibility

I would make the dedupe/cache key use an **authorization fingerprint**, with the operation declaring whether identity itself participates in that fingerprint.

### 3. "Never stored" is not the same as "never cached"

For learner PII, saying:

> `ttlSeconds = 0`: always fresh, never stored

is technically misleading if the result still exists on the queue result row or in blob storage.

The accurate statement is closer to:

> **Never cached. Transient job results remain subject to queue-result retention.**

That distinction will matter in a security review.

### 4. Lease expiry still permits duplicate execution

The lease model allows this:

```text
Worker A claims job
Worker A stalls past lease
Worker B reclaims job
Worker A wakes up
Both execute
```

The in-process single-flight does not prevent that because they are separate processes.

For read-only operations, I think that is fine.

But we should describe the guarantee accurately:

> **At-least-once execution, exactly-one durable result.**

We should not imply that single-flight eliminates the lease-lapse window globally.

### 5. Heartbeat and lease timing

A 60-second heartbeat against a 120-second lease feels tight.

One missed heartbeat leaves very little tolerance.

I would either heartbeat around 30 to 40 seconds or make the lease something like 180 seconds.

A dead worker costing us another minute is less damaging than a healthy worker being falsely reclaimed.

### 6. Do not make "one worker per surface" a religion

I am less convinced by one worker type per surface than I am by the rest of this architecture.

Twenty-four surface worker types may be perfectly reasonable today.

But a surface is fundamentally a UI organizational boundary. It does not necessarily need to remain a compute boundary forever.

I would absolutely keep the queue type abstraction, but I would not make us philosophically committed to one container deployment per surface.

Eventually several lightweight surfaces may share a worker fleet while heavier surfaces retain dedicated pools.

That is an implementation choice we should be able to change without disturbing the contract above it.

## Where this gets us

Today we effectively have:

```text
Front end
   ↓
Lots of queue implementations
   ↓
Lots of workers
```

We are moving toward:

```text
Browser
   ↓
Surface BFF
   ↓
Louie
   ↓
Whatever can do the work
```

And once we get there, the caller does not care whether the thing doing the work is Enrollment, SCORM, Maestro, Node, .NET, Postgres, an AI worker, or something we have not invented yet.

That is really what Louie is.

**Louie is the contract between "I need something done" and "something knows how to do it."**