Before retiring an old tool, the company checks whether the replacement makes the same decisions.
First, the team records the situations handled by the old system and the expected result for each one. The same cases then run in the new environment. Every difference goes to the process owner, and the return path is tested before cutover.
What we measured
The reference implementation shows the full migration process: 336 behavior checks, seven unresolved differences, a test of the new environment, and a return rehearsal. It does not connect to the client's system or the old production environment.
- Test in the new environment
- What we measured
- Measurement scope
- Moving an automation to a new environment
- Verified
- synthetic data
Problem, solution, and result
The problem
A critical automation may run on a platform that is being retired, while many of its rules remain undocumented. Rebuilding the visible flow does not guarantee that exceptions and decisions will behave the same way.
How it works
The team first records the expected response for representative cases. It then runs the same set through the replacement and logs every difference. Cutover can proceed only after those differences are accepted or fixed.
What the recorded run measured
The reference implementation shows the full migration process: 336 behavior checks, seven unresolved differences, a test of the new environment, and a return rehearsal. It does not connect to the client's system or the old production environment.
For whom
This migration method fits a critical process running on a retiring tool whose rules and exceptions have never been fully documented.
Catalogue → comparison → trial → return
- 01The new environment answers for 336 behaviors of the old one, case by case
- 02A discrepancy without a decision holds the cutover and traffic stays on the source
- 03The way back is rehearsed before cutover, and open differences go to the owner's review
- Business type
- Companies maintaining production automations in an older environment
- Input
- A fixed behavior set: 336 cases with inputs and expected decisions
- Boundary
- A discrepancy without a decision holds the cutover; traffic stays on the source
- Cost
- USD 0.030373 for the recorded 12-case run, or about USD 0.002531 per case.
- Security
- An unresolved difference stops the switch. The team can keep traffic in the existing environment.
- Speed
- We measure switch and return time during a rehearsal on the client’s infrastructure.
- Trace
- The history shows compared behavior, accepted differences, the switch decision, and the return rehearsal.
- Similar system build
- from €6,000 net (approx. $6,500) · 6–16 weeks
Reference exchange rates: USD 1 = PLN 3.72; EUR 1 = PLN 4.30. PLN amounts are rounded, with the measurement currency retained in parentheses.
The cutover boundary
An unresolved difference stops the migration
A discrepancy without a decision holds the cutover. Each of the seven open items goes to the automation owner, and traffic stays in the existing environment until the issue is resolved. The demonstration can only look up an order and rejects any attempt to promise a shipment or change data.
- Boundary
- A discrepancy without a decision holds the cutover; traffic stays on the source
- Cost
- USD 0.030373 for the recorded 12-case run, or about USD 0.002531 per case.
- Security
- An unresolved difference stops the switch. The team can keep traffic in the existing environment.
- Speed
- We measure switch and return time during a rehearsal on the client’s infrastructure.
The seven differences we did not close
Seven unresolved differences still block the full switch in the demonstration. The return procedure worked once in a local test, so production reliability needs repeated rehearsals. Response time and operating cost must also be measured again under the client’s actual traffic.
Estimated impact
Calculate the impact at your volume
This estimate uses the stated volume. Enter your own numbers to assess the possible impact in your company. The result is confirmed only during a pilot.
Today
Assumption: 120 days/program (8 × 15 days)
With the system
Scenario: 72 days/program (8 × 9 days)
Time or cost saved
Modelled result: 36–60 days, base 48
- Volume
- 8 agents
- Formula
- 8 × (15 − 9) = 48 days/program
- Calculation status
- low
Data in the screenshots. The names, amounts, and documents shown in the screenshots are synthetic. Client data remains private. Measurements describe the demonstrator and are kept separate from production outcomes.
Working surfaces
Review on the owner's side settles the open differences and approves the cutover.
The automation owner opens one view and reads what nobody knew before: how many behaviors match, who owns each open difference, and whether the return path works. They walk into the cutover conversation with a file, and the seven open items sit in it as a visible block waiting on a decision. On client data, behavior under production traffic and the running cost of the new environment still have to be measured.
The migration file: status, difference report, cutover gates.
Migration status
The golden set, the recorded eval result, the discrepancy count, and the return-path status in one view.
Parity report
Every difference with its classification and either a review decision or an explicit open status.
Cutover gates
A runbook from the source inventory to the rollback trial, with each gate's state stated plainly.
System screens
See how the system works in practice
These desktop and mobile screenshots come from the running application. They show the process and the points where a person makes the decision.
- Screens
- 13
- px
- 1440 · 390
- 021440×3517
The recorded target-runtime eval: 12/12 contracts, 36 assertions, and 12/12 verified traces. - 031440×1100
Mobile view of migration status.
Open the remaining screen archive (10)
- 041440×1100
Decision field and owner signature - 051440×1100
Register of 48 divergences - 061440×1100
Catalogue of 336 behaviors - 071440×1100
Source inventory - 081440×1100
Verified target environment measurement - 091440×1100
Full test record, matching case - 101440×1100
Full test record, divergent case - 111440×1100
Parallel observation: zero state - 121440×1100
Cutover card: held - 131440×1100
Measured local rollback
Technology stack
A target runtime on a supported SDK, with ordinary inspectable engineering around the comparison.
A migration does not need a new platform. It needs an environment you can hold to account for its behavior, so we use the supported OpenAI Agents SDK as the target, typed contracts on the way in and out, and a database that remembers every decision. Six containers separate the web app, the API, the database, both environment pointers, and the cutover router, so there is exactly one explicit switching point, and the source stays a frozen reference. For maintenance that means each part can be checked on its own, and the golden set stays behind as the test suite for whatever changes next.
- OpenAI Agents SDK 0.7.0
- the target agent runtime: intent classification, one read-only tool, a validated result
- FastAPI + Pydantic
- publish the inventory and decision contracts and reject answers outside the allowed scope
- PostgreSQL 17 + pgvector
- stores the inventory, the review decisions, and the audit chain of every run
- Cutover router
- the one explicit switching point, with a return to the source that was executed and recorded
- Next.js 16 + React 19
- serve the migration file: status, parity report, and cutover gates
The client takes over the code, the prompts, the golden set, and the documentation; they came for a migration and leave with the tests they never had. The reference implementation runs in Docker Compose on generated test data and has no connection to either client systems nor the source runtime.
Technical details and measurements
The working loop
The approved catalogue defines correctness, and the new environment has to answer to it.
A case-by-case comparison turns the hunch that it answers similarly into a list of matches, discrepancies, and decisions. That list lives in the file, together with everything still unresolved.
A frozen catalogue: 336 expected behaviors with their decisions
Case-by-case comparison: every difference decided or visibly open
New-environment trial → intent, review gate, read-only scope
Cutover trial → return to the source
Traffic stays on the source until every gate is closed
System architecture
A frozen catalogue, a parity report, a recorded eval, and a router with a return path.
Four parts, each inspectable on its own: the catalogue of expectations, the comparison with review decisions, the eval on the target SDK, and a router whose return path has already been executed and recorded.
- 01
Snapshot
Expected behavior becomes the golden set.
336 prepared synthetic cases with inputs and expected decisions form an explicit definition of correctness. This is the step migrations usually skip, and without it the sentence "it behaves the same" means nothing.
- 02
Parity
Every discrepancy is counted, to the last one.
48 discrepancies went through review: 41 carry a closed decision, 7 remain open and stand in the file as a block on full cutover.
- 03
Recorded eval
The target runtime gets a controlled sample.
Two synthetic cases for each of six intents run through the OpenAI Agents SDK. The eval checks intent, review routing, and read-only scope, then records the result and the trace.
- 04
Router
Cutover with a return path that was performed.
A separate trial flips the router to the new inventory, confirms the target, and returns to the source in 2.086 ms. The rollback was executed and recorded, so it is not a plan on paper.
Why so little machinery
The target agent consists of intent classification, one read-only tool and a validated result. Keeping the migration stack small makes behavioral parity easier to verify.
- The golden set defines correctness before the change
- A discrepancy without a decision holds the cutover
- The eval carries review routing and a read-only tool
- The rollback was executed and recorded
Want to assess a similar process in your company?
- A 30-minute call with the engineer who would lead the work.
- A review of the processes that cost you the most time and money.
- A written summary of what to automate first and the likely cost range.
Times are shown in your own time zone. We work with clients across time zones.
The scan is free and creates no obligation. If automation is unlikely to pay off, the written recommendation will say so.