Claude Code meets the mainframe

COBOL still runs much of the world's banking, insurance and payments. The estates behind it were built over decades, by people who have mostly moved on, and they no longer fit in anyone's head: programs by the hundred or thousand, copybooks shared across many of them, values handed on through files, screens, queues and tables.

So before anyone changes a line, someone has to answer a question. What does this program do? Who calls it? If this field grows by one digit, what breaks? On a large estate that answer can take a senior developer hours or days, and a wrong one reaches production.

The links that matter live where a text search doesn't connect them: a program called through a table of names, a dataset named only in a JCL step, a field moved into a commarea and read three programs later. Alone, an agent finds them the way a newcomer would, opening one file at a time.

Replai indexes the estate first: every program, copybook, job, file, table and screen, and how values move between them. Claude Code reaches that index through one MCP server. We wanted to know what that changes on real code, so we measured it on two estates of very different size.

Replai reads every source in the estate once and builds one index of it: what exists, how it connects, what the data looks like and where each value travels. Every fact keeps the file and line it came from, so every answer can cite them.

Claude Code reaches the index through the Model Context Protocol. It asks in the terms developers use, and gets back short, cited facts instead of a pile of files to read.

Without Replai, Claude Code answers an impact question by grepping, opening a file, grepping again, opening the next. With Replai it asks the estate once, gets every includer, caller, file and record length with a line citation, and opens only the lines it needs to confirm.

Each question was answered twice by Claude Code on its own and twice with Replai, then graded blind against a reference answer built from the source. Figures are averages per question.

On the 73,000-line public estate, Claude Code alone does respectably: the code is small enough to search, and public, so the model may well have met it before. On about 2 million private lines, searching gets slow and answers get thin. That is where the map pays most: a search costs more the bigger the estate, while a lookup in the map hardly does.

Each of these gains grew with the estate. On the large one Replai was faster and much more accurate, and the cost gap was wider.

We asked sixteen questions of each kind on each estate. The examples below are real ones from the public estate.

Most of what an agent spends on a legacy estate goes on reading: search results, whole programs, copybooks opened to check one field. Replai does that reading once, when it builds the map, and each answer arrives as a short brief with a citation on every line. In this benchmark that alone cut tokens by 35% and cost by 47%, with the same model on both sides.

It also changes which model you need. Once the map has done the cross-referencing, the model's job is to reason over a few hundred cited lines instead of searching millions, which is work smaller models do well. The index lets teams run an even cheaper model and keep the view of the whole estate.

The same map makes generated code far more accurate. A model that edits the file in front of it gets that file right and misses the rest: the programs that copy the record, the commarea that grows by a byte, the Db2 column that silently drops a digit.

Replai's change workflow starts from the whole estate. It finds every program a change reaches, plans the edit across all of them for a person to approve, applies it in place, then compiles and tests the result.