Why vendor evaluation goes wrong on legacy projects
Most modernization projects that stall do not stall because of the new technology. They stall because nobody fully understood what the old system did before work began. Teams spend months trying to work out what the existing code does before writing anything new, and some projects fail outright or deliver a fraction of what was promised because critical business rules stayed buried and got lost in translation.
That is the lens to use when you compare vendors. A polished demo on a clean sample application tells you little. The questions below are built to separate a vendor that can handle your real system, with its missing documentation and departed authors, from one that is selling a general promise. Use them in a live call, not a questionnaire, and ask for a demonstration on a slice of your own code wherever you can.
Questions about platform and language support
- Which languages and platforms have you handled in production, and which ones is this demo built on? Legacy estates mix COBOL, PL/I, aging Java and Oracle Forms. Ask which of those the vendor has actually worked on, not which ones appear on a slide.
- Can you run against a portion of our real code? A pilot on your own modules shows how the tool behaves with undocumented customizations and shared libraries. A sanitized sample will not.
- What happens with code you do not support? Every estate has odd corners. A good vendor can tell you where the tool stops and what the plan is for that remainder.
- What is the target architecture, and who chooses it? For Oracle Forms, teams commonly move to Oracle APEX because it keeps the Oracle Database relationship and reduces the rewrite surface for PL/SQL logic. Others rebuild in Java, Angular or React. Ask whether the vendor is tied to one destination or can justify the choice for your situation. Our Oracle Forms migration guide walks through that decision.
Questions about business logic accuracy
This is where marketing claims and real capability diverge the most. Reading syntax is easy. Connecting code to business meaning is hard, and it is the part that decides whether a rewrite preserves your rules.
- How do you extract business rules, and how do you show they are correct? Ask to see the output for a rule your own team already knows well, then check it against what they expect.
- Can every finding be traced back to the source code? If a summary cannot point to the exact trigger, paragraph or procedure it came from, your reviewers cannot verify it.
- Where does the tool say it is unsure? A vendor who claims perfect extraction on decades of accumulated logic is overselling. You want a tool and a team that flag ambiguity instead of guessing quietly.
- How much of the discovery work is automated, and how much still needs your experts? Manual inventory of a large estate often eats the first months of a project. Ask for a realistic split, and read our explanation of how AI agents read legacy code so you know what a credible answer sounds like.
- What does your output look like when the project ends? You want something your own engineers and analysts can read, not only generated code.
Questions about security and data handling
Legacy code in banking, insurance and logistics often carries regulated logic and sometimes embedded data. Treat security as a gate, not a follow-up item.
- Where does our code go during analysis? Ask whether it is processed in your environment, the vendor's environment, or a third-party model provider, and get the answer in writing.
- Is our code or output used to train any model? The answer should be a clear no, with contract language to match.
- Who on your side can access our source, and how is that access logged and revoked? Ask for the access model, not a general assurance.
- What documentation can your security team hand over to ours? A vendor that treats enterprise security and privacy as foundational should have a trust center or equivalent material ready before you ask.
Questions about knowledge handover and what you keep
- What do we own when you leave, and in what form? Extracted rules, maps of how code components relate, and decisions made along the way should be yours, readable without the vendor's tool or staff.
- How does your work reduce our dependence on a few people? Many estates lean on a handful of experts close to retirement. The best outcome is that the understanding now lives in an artifact, not in someone's head. Our page on legacy system knowledge transfer covers what to capture and why the timing matters.
Two related points to settle before you sign: how pricing is structured against scope changes, since discovery surprises are what usually blow budgets (see legacy system modernization cost), and whether the vendor supports an incremental path instead of forcing a full replacement (see AI-driven modernization vs. full system replacement).
Score each vendor on the same list, weight accuracy and handover most heavily, and insist that the strongest claims are demonstrated on your own code before any contract is signed.