Give the Agent Hands: Tools in Embabel

Every agent I'd built in Embabel up to this point only did one thing: ask the model to generate text. Useful, but limited — the model can only reason over what's in the prompt. The moment you need live data, generation isn't enough. An LLM cannot know the balance of account ACC-100; it lives behind your walls. Ask it anyway and it will invent a plausible number, which in a banking context is not a bug, it's an incident. The fix is a tool: a real function the model can call mid-generation.

A tool is just an annotated method

In Embabel a tool is an ordinary method annotated @LlmTool, with @LlmTool.Param on its arguments. The descriptions aren't documentation — they're the interface the model reads to decide whether and how to call it. Here's a tiny in-memory balance store standing in for what would really be a repository or a downstream service:

public static class AccountTools {
    private static final Map<String, Integer> BALANCES = Map.of(
            "ACC-100", 1250, "ACC-200", 340, "ACC-300", 9800);

    @LlmTool(description = "Look up the current balance in EUR for an account id. Returns -1 if the account is unknown.")
    public int getBalance(
            @LlmTool.Param(description = "Account id, e.g. ACC-100") String accountId) {
        return BALANCES.getOrDefault(accountId.trim().toUpperCase(), -1);
    }

    @LlmTool(description = "List every known account id")
    public Set<String> listAccounts() {
        return BALANCES.keySet();
    }
}

Notice there is nothing framework-y inside these methods. They're plain Java. That matters for testing, which I'll come back to.

Attaching tools to an action

You hand the tools to a specific LLM call inside an @Action. This is where I hit the one snag worth writing down. Following an older docs example, I wrote ai.withToolObject(...) and got:

cannot find symbol
  symbol:   method withToolObject(...AccountTools)
  location: variable ai of type com.embabel.agent.api.common.Ai

The chain is a small state machine, and the order encodes intent. Ai answers one question — which model? Only after withDefaultLlm() (or withLlm(...)) do you hold a PromptRunner, and that's where per-call configuration lives: tools, tool groups, prompt contributors. Tools are scoped to a single invocation, not the whole agent, so they attach after model selection, not before:

@AchievesGoal(description = "The customer's account question has been answered using looked-up balances")
@Action
AccountAnswer answer(UserInput userInput, Ai ai) {
    var prompt = """
            You are a retail-banking assistant. Answer the customer's question.
            You MUST use the provided tools to look up real account balances --
            never guess or invent a balance. All amounts are in EUR.
            If the customer asks whether a balance covers an amount, do the arithmetic.

            # Customer question
            %s""".formatted(userInput.getContent()).trim();

    return ai
            .withDefaultLlm()                   // Ai -> PromptRunner
            .withToolObject(new AccountTools())  // per-call: hand the LLM the tools
            .createObject(prompt, AccountAnswer.class);
}

I found the correct signature not by guessing but by grepping the dependency's own sources: withToolObject is declared on PromptRunner, alongside withToolGroup, withToolObjects, and friends. When docs and reality disagree, the jar wins. A five-second test-compile beats any amount of runtime debugging.

What actually happens at runtime

Earlier in my logs, every run printed starting tool loop [] max=20. That empty [] was the list of available tools — there weren't any. Attaching a tool object fills it. Now the model, mid-generation, can emit a call to getBalance("ACC-100"); Embabel runs the real method, feeds 1250 back, and the model continues — up to 20 iterations of that loop. The output stops being a guess and becomes grounded in a value my code returned. That's the whole point of the pattern: the model decides when it needs data; my code decides what the data is.

The testing payoff

Because the tool is plain Java, its logic is testable with plain JUnit — no Spring, no fake LLM, nothing:

private final AccountAssistantAgent.AccountTools tools = new AccountAssistantAgent.AccountTools();

@Test void returnsBalanceForKnownAccount()   { assertEquals(1250, tools.getBalance("ACC-100")); }
@Test void isCaseAndWhitespaceInsensitive()  { assertEquals(340,  tools.getBalance("  acc-200 ")); }
@Test void returnsSentinelForUnknownAccount(){ assertEquals(-1,   tools.getBalance("ACC-999")); }

Four assertions, 40 ms, zero tokens. This is the same separation-of-concerns theme that runs through everything in Embabel: keep the deterministic part (what the balance is, how ids are normalized) in ordinary code you can hammer with unit tests, and let the non-deterministic part (does the model choose to call the tool, how does it phrase the answer) be the only thing that needs a live model. Your integration test then only has to prove the LLM reaches the tool, not that the tool computes correctly.

Why this is the important pattern

Text generation is where agents start; tool use is where they become useful. A tool is the seam between the model and your system — the database, the pricing service, the ledger. Get it right and you get grounded, auditable answers with the arithmetic done against real figures. Get it wrong — skip the tool, let the model "know" the balance — and you've shipped a confident liar. In regulated work that distinction is the whole job. In Embabel the seam is one annotation and one line on the PromptRunner, and everything behind it stays boring, typed, and testable. Exactly how I want it.