Testing a Claude Code plugin against no plugin at all
I built a small commit-message plugin and ran claude plugin eval on it. The no-plugin baseline showed what the plugin actually adds, and the one red case turned out to be the judge's mistake.
Tag
I built a small commit-message plugin and ran claude plugin eval on it. The no-plugin baseline showed what the plugin actually adds, and the one red case turned out to be the judge's mistake.
I ran Anthropic's code-modernization plugin for Claude Code headless on a 2006-style billing batch I had booby-trapped. It caught every trap, found four problems I hadn't planted, and still only called its own proof "partly proven".
Scala and the JVM · Part 15
I unpack AnyFunSuite from a Java developer’s perspective, test returned Right and Left values, check thrown exceptions with intercept, and verify that independent expectations catch real mistakes.
Building agents on the JVM with Embabel · Part 2
The non-deterministic part of an agent is the LLM. Box it out and the rest is ordinary, deterministic code you can test. Here's how I test Embabel agents on the JVM without spending a single token.