The second run is cheaper: what an Embabel Context actually buys you

Here's a thing that happened while I was learning Embabel that I didn't design and didn't expect: I asked my agent the same question twice, and the second time it answered better and called the model less. Not because I added a cache. Because the answer was already there, and a planner that works backwards from state doesn't schedule work whose output already exists.

The setup

A small agent over a Postgres orders table. Three actions, and nothing anywhere declares the order they run in:

CustomerName extractCustomerName(UserInput userInput, Ai ai)   // an LLM call
CustomerOrders loadOrders(CustomerName customerName)           // a SQL query
CustomerInsights buildCustomerInsights(CustomerName n, CustomerOrders o)  // @AchievesGoal

Parameters are preconditions, return types are effects, and Embabel's GOAP planner runs an A* search over that to produce the chain. It also branches: if the name matches no orders, a second action asks the model which known customer was probably meant.

Sessions persist because I replaced the framework's in-memory ContextRepository with one backed by Postgres. A Context is a bag of domain objects that gets copied onto a new process's blackboard before planning starts, so restoring a session is just restoring that bag. Wiring it to a run is one line:

AgentInvocation.builder(agentPlatform)
        .options(ProcessOptions.DEFAULT.withContextId(session))
        .build(CustomerInsights.class)
        .invoke(new UserInput(ask));

Run one: two model calls to work out who I meant

The seed data has Zaphod Beeblebrox. I asked about Zaphod.

orders "how many did Zaphod order?" --session l2b
> No orders found for customer 'Zaphod'. Did you mean 'Zaphod Beeblebrox'?

Three actions, two model calls. One to pull "Zaphod" out of the sentence, then — after the lookup came back empty and flipped the condition — one more to match it against the known customers. Just over a second end to end.

A correct answer, and a completely useless one to repeat. The interesting question is what the agent should remember.

The write-back is the whole design decision

Nothing copies the blackboard back into the Context for you. That's application code, and the obvious version is wrong:

context.addObject(new CustomerName(insights.customerFullName()));  // stores "Zaphod"

customerFullName() is the name I asked with — the one that found nothing. Persist that and the next run restores it, skips extraction, looks up nothing, suggests Zaphod Beeblebrox again, and stores "Zaphod" again. The agent computes the right answer every single run and throws it away every single run. I watched it do this three times before reading the log properly:

Context 'l2b' AFTER run -> [CustomerName[fullName=Zaphod],
                            CustomerName[fullName=Zaphod],
                            CustomerName[fullName=Zaphod]]

What you want to persist is the conclusion, not the input that failed to resolve:

var suggestion = insights.suggestion();
var remembered = suggestion != null && !suggestion.isBlank()
        ? suggestion                      // "Zaphod Beeblebrox"
        : insights.customerFullName();

// createWithId replaces the stored bag; reusing the loaded Context appends to it
var context = contextRepository.createWithId(session);
context.addObject(new CustomerName(remembered));
contextRepository.save(context);

Run two: the step that isn't in the plan

Same command, same session id. The Context now puts CustomerName("Zaphod Beeblebrox") on the blackboard before the planner runs. So when A* searches for a route to CustomerInsights, the state extractCustomerName would have produced is already satisfied.

It isn't pruned as an optimisation. It is never a candidate. The plan is two steps — lookup, then build — and there is no model call in it at all: the lookup now hits a real customer, customerIsKnown comes back true, and the branch that needed a model to guess is never reached either.

Two model calls became zero, three steps became two, and the answer changed from a suggestion into the actual order count. The agent resolved the name once and never had to think about it again.

Why this isn't a cache

A cache is a lookup you write, keyed by something you choose, checked at a point you decide. None of that exists here. There's no key, no hit-or-miss branch, no invalidation, no code path that says "if we already know the name, skip ahead". The only thing that happened is that a value of the right type was present, so an action that produces that type had nothing left to contribute.

Which is the actual argument for type-driven planning, and it took me writing the bug to see it. In a framework where you hand-write the flow, "don't re-extract the name if we already have it" is a conditional you have to remember to add at every call site, and it rots the moment someone inserts a step. Here it's a consequence of how the plan is derived. I got it without asking for it — and, less comfortably, I would have got the broken version without asking for it too.

Because that's the trade. The optimisation and the bug are the same feature. "Skip the action whose output already exists" is what makes the second run free, and it's exactly what made the wrong persisted name permanent. There's no flag separating them, because the planner genuinely cannot tell whether a value on the blackboard was computed this run, restored from Postgres, or wrong. It only knows the type is present.

What I'm taking away

Deciding what to persist is design, not plumbing. Everything written into a Context is asserted as settled fact for every future run in that session, and it silently disables the action that would have produced it. Store conclusions. Never store an input you failed to resolve — if a value might be wrong, it's cheaper to let the agent recompute it than to be trapped by it.

Measure the plan, not the output. Both runs printed something reasonable; the difference was in plan length and call count. Logging the Context before and after each run is four lines and is the only reason I noticed any of this.

The blackboard is the API. I keep reaching for control-flow thinking — "then it does this" — when the useful question is always "what's on the blackboard, and what does that make reachable". Every surprise Embabel has handed me so far has come from answering the first question instead of the second.