Java 8 to 21 on a batch full of traps I planted: the code-modernization plugin, end to end

Anthropic ships a Claude Code plugin called code-modernization. It walks a legacy codebase through assessment, mapping, business-rule extraction, a signed plan, the build, an independent proof and a security pass. I wanted to know whether it notices the behaviour a careless upgrade breaks, so I gave it a system where I knew where every trap was, because I had planted them.

Everything below is from one session: a synthetic legacy system that I wrote for the test, the plugin running headless one command at a time, and me in the role of system owner at each human gate.

The patient: a nightly billing batch from 2006

"Dirt Cheap Equipment Rentals" is about 800 lines of Java written in the style of the time: Vector, Hashtable, raw types, new Double(...), a finalize(), commons-lang 2.6 and log4j 1.2.17, built with Maven for Java 8. Every night it prices returned rentals and writes fixed-width invoices for a dot-matrix printer, a ledger.sql for accounting and a summary for the yard manager. The goal I gave the plugin was Java 21, current plugins, JUnit 5, and behaviour must match exactly, including known quirks.

The quirks are what made this a fair test:

• Money is a double rounded with Math.round(v * 100) / 100.0. Oklahoma's 4.5% on $105.00 is 4.7249999… as a double, so the invoice says 4.72 where decimal arithmetic says 4.73. The README calls this "1 cent off the calculator now and then".
• Rental hours are elapsed milliseconds divided by 3,600,000. A rental across the March DST change loses an hour, which turns one contract into 1 billed day. LocalDateTime arithmetic would bill 2.
• Invoice numbers follow Hashtable iteration order. Swap in a HashMap and every invoice is renumbered.
• The damage waiver is computed on the rental before the loyalty discount, which was an accounting decision.
• For the security pass: SQL built by string concatenation (a customer called O'Brien breaks it), a hardcoded database password echoed into every ledger.sql, a default DocumentBuilderFactory, and file names built from customer names.

Driving it headless

Each plugin command is a slash command, so I ran each one as its own non-interactive session from the workspace folder and logged the stream:

claude -p "/code-modernization:modernize-assess dirtcheap" \
  --permission-mode auto --output-format stream-json --verbose > logs/02-assess.jsonl

That only works because the plugin keeps its state in files rather than in the conversation. INTENT.md, PREFLIGHT.md, BUSINESS_RULES.md, RULE_REVIEWS.json and MODERNIZATION_BRIEF.md are what the next command reads, so a fresh session picks up exactly where the last one stopped. Legacy code lives under legacy/ and is never edited. The plugin's own preflight recommends a deny rule for that, so I added Edit(/legacy/**) to the project settings. It later blocked one of my own shell commands that resolved a relative path into legacy/.

The human gates do not go away when you run headless. The review command says outright that, with no pop-up available and no typed verdicts, it must "not guess: leave every rule unreviewed". So I typed the owner's verdicts into the prompt. The brief has an approval block that I signed by editing the file, because the brief itself says an edited criterion is binding and a chat message is not.

What it caught

assess found all three output traps and filed them under cleanups that would break output: Hashtable to HashMap, double to BigDecimal, and elapsed time to LocalDateTime. extract-rules fanned out 84 agents and produced 33 Given/When/Then rule cards, each citing file:line. A referee panel rejected none of them and merged 29 duplicates. The cards are specific enough to argue with. One notes that because hours are truncated before the 2-hour grace test, the effective grace is just under three hours.

The more interesting part is what it found that I had not planted:

• JEP 400. Every FileReader and FileWriter relies on the platform charset, which is UTF-8 by default from Java 18. A customer called Peña would produce different bytes in every output file after the upgrade. Preflight flagged this before anything else had run.
• CRLF on my machine. The brief noticed that a golden master recorded on Windows would freeze the wrong line endings for a Linux batch server.
• Double.toString in the ledger. The uplift's fixtures include an invoice of exactly $10,000,000. It is written to ledger.sql as 1.0E7, and that is now pinned by a test.
• A real data-loss bug. Invoice files are named after the customer's name, so two accounts with the same name overwrite each other's printed invoice while both still post to the ledger.

The uplift itself was small

The diff to src/main is about 35 lines plus one new class. Readers and writers name ISO-8859-1 explicitly (the yard server's charset). finalize() and the boxing constructors are gone. commons-lang moved to commons-lang3, and log4j 1 was replaced by reload4j. Line endings are pinned with a 21-line subclass:

public class LfPrintWriter extends PrintWriter {
    public LfPrintWriter(Writer out) { super(out); }

    /** every println(x) ends with this */
    public void println() { write('\n'); }
}

The work went into the evidence. The pilot recorded eight golden-master sets from the legacy jar, including Latin-1 names, a DST fall-back night, a set of 20 customers sized to make the Hashtable resize, and a timestamp inside the spring-forward gap that the strict date parser rejects. The build runs them under three JVM configurations: host defaults, yard settings, and German locale with Tokyo time. It then reverted its own charset fixes to check that the tests fail. When I re-ran the build myself, it passed 34 of 34 tests.

Verify said "partly proven", and I agree

verify re-ran the suite, ran canaries (changing the grace hours from 2 to 3 made three tests fail), and invented 13 fresh inputs for 276 byte-identical comparisons. It still would not say PROVEN, because the one legacy test file had changed. That change was only the JUnit 4 to 5 port, but its rule is that edited tests prove less, and deciding whether that is acceptable belongs to a person. I signed with a condition.

The condition is the honest gap in this run. There was no JDK 8 here, because the download failed certificate validation on my network and I did not switch the check off. Both sides of every comparison therefore ran on JDK 21. The plugin labelled the result "target-only" throughout and kept a Java 8 vs 21 shadow run on the real server as the step to do before any cutover.

Harden, and a patch aimed at the wrong tree

harden ran 5 finders, 30 refuters and 5 second judges. They dropped 6 of 30 raw findings as false positives and kept 18. Besides the SQL injection and the leaked password, it found one I had not intended: split(",") on the customer file means a name like Acme,G,DE,Y gets Gold pricing and no tax. The patch it produced targets legacy/, which makes sense because harden can run straight after assess. After an uplift, though, the fix belongs in the new tree. I ported it myself, re-recorded only ledger.sql in the golden sets, and checked that every invoice, summary and reject file still matched its recorded SHA-256. My first attempt at that check reported everything as mismatched. The cause was sha256sum escaping its output for a Windows path with backslashes, not the code. Hashing from stdin fixed it.

What I take from it

The whole run took 106 minutes of agent time and $45 at reported API prices, for an 800-line system. Reading the code cost more than changing it, and I think that is the right ratio. The useful output is not the 35-line diff. It is BUSINESS_RULES.md, the delta catalog and a test suite that encodes the system's odd behaviour as something to keep.

Two practical notes if you script it. First, background sub-agents die when a claude -p process exits. My first brief run launched its delta analyst in the background, and the agent was killed when the run finished. Resuming the session with an instruction to work in the foreground fixed it. Second, the plugin leaves scratch directories in C:\tmp and keeps credential-bearing files in ~/.modernize/, outside the workspace. It tells you so, but you have to clean up after it yourself.

The plugin did not try to make the batch better. It tried to keep it the same and to show, with evidence, where it could not prove that. For a version upgrade, that is the right priority.