Blog · Published · Updated
Runtime comparison: how the workflow works
Short answer
You move an agent to a new environment after comparing it on a golden set: the same inputs, the expected decisions and a process owner's decision on every difference. In our demonstrator the set holds 336 behaviours, and 7 unresolved differences block the switch. A full run of the set costs about €0.76.
How we workDetails ↓
- 01Call30 minutes with an engineer, free
- 02PrototypeA working system on your process and data, free
- 03ImplementationTarget and fixed price in the contract; 50% at signing
- 04Acceptance50% once the agreed target is met
- analysis with an engineer and a free prototype
- €0
- team in ChatGPT or Claude, up to 10 people
- from €1,150
- automation, agent or application for one process
- from €3,000
We belong to the OpenAI and Anthropic partner programmes and learn through their official courses. What this means ↓
Author: Syntalith · 3 min read
Statistics
Agent migration: statistics 2026
7
unresolved differences block the switch in the Syntalith demonstrator, with a golden set of 336 behaviours. Traffic stays in the old environment until a decision.
336
According to the Syntalith demonstrator (2026), the golden set for an agent migration holds 336 behaviours, and 7 unresolved differences block the switch to the new environment.
$0.002531
According to the Syntalith demonstrator (2026), a recorded run of 12 cases in the new environment cost $0.030373, about $0.002531 per case.
40%
According to Gartner (2025), over 40% of agentic AI projects will be cancelled by the end of 2027 because of costs, unclear value or weak risk controls.
90%
According to the European Commission (2026), 90% of companies using AI say people take the decisions, and 2% of decisions are fully automated.
Source: European Commission, Flash Eurobarometer 578 (2026) ↗
Figures checked: 22 October 2026
What changes when you move an agent
A critical automation often runs in a tool that is about to be switched off, and its rules are not fully written down. Rewriting the flow alone gives no certainty that exceptions and decisions stay the same. The same case can end in a different decision after a change of environment, so the comparison covers tool calls and decisions. A similar final answer is not enough.
- 01Set336 cases with inputs and expected decisions
- 02Runthe same set in the new environment
- 03Differencesevery difference goes to the process owner
- 04Decisionan accepted difference or a fix
- 05Rollbackrehearsed before the switch
A difference without a decision stops the switch
A difference without a decision stops the switch whatever the overall match percentage. In the demonstrator seven open items wait for a decision by the person responsible for the automation, and traffic stays in the old environment. The history shows the compared behaviours, the accepted differences, the decision to switch and the rollback test. Hugging Face, in its notes on evaluating models, reminds you that an evaluation set must reflect real use, so we build the golden set from real cases, rare ones included.
- Full run of the set
- 336 × $0.002531 = $0.85 (€0.76)
- Reviewing the differences
- 7 × 30 min = 3.5 h × €34.90 = €122
One comparison with review: about €123
The cost of the run comes from the Syntalith demonstrator ($0.030373 for 12 cases), converted at the ECB rate of 1.1205 USD per euro on 22 October 2026. The time to review one difference is the scenario's assumption. €34.90 an hour is the EU average hourly labour cost for 2025 according to Eurostat.
The cost of a comparison is small next to the cost of a changed decision in production, so we rerun the set before every change of model, tool or instructions.
How to build an agent migration golden set
The set starts with the situations the old system handles today: ordinary cases, exceptions, refusals and cases where the system hands over to a person. For each case you record the input, the expected tool calls and the decision. Rare cases get their own entries, because that is where the new environment most often behaves differently. You agree the criteria for accepting a difference before the first run, together with the condition for rolling back to the previous version. According to Gartner (2025), over 40% of agentic AI projects will be cancelled by the end of 2027, partly because of unclear value and weak risk controls.
What the demonstrator does not prove
- Seven open differences still block a full canary
- The live trial covers intent recognition, the operator's control boundary, reading an order and writing the audit record
- The old runtime did not record token use, so the cost ratio was not calculated
- One local rollback does not prove reliability under load
The public version runs on test data and connects neither to a client system nor to the old production environment. We measure switch and rollback times in a trial on the client's infrastructure.
The first migration in a company
Simple rules in a tool being retired
a rewrite with tests on known cases
from €3,000
An agent with decisions and exceptions
a golden set, a review of differences and a rollback plan
an agent from €3,000
Many agents and integrations
a migration in stages, each with its own set
priced after the review
To prepare: examples with an agreed correct result, a decision owner with acceptance criteria, and data sources with a defined scope of access. We run agent migrations as part of AI agents, in 6 to 16 weeks. For the free process review, a description of the automation that has to move and the tool that is being switched off is enough.
Worked example · Scenario
How we count the payback: an IT team
We run the same count for your automation in the free process review.
€4,188
a month for searching the documentation for answers to 600 questions
Payback: 0.8 mo.
12 minutes to find an answer or ask a colleague and checking in 15% of the old time are our assumptions. €34.90 an hour is the EU average hourly labour cost for 2025 according to Eurostat, for companies with 10 or more employees. Model cost: Claude Sonnet 5.5, about 6,000 + 500 tokens per answer from the documents, at the ECB rate of 1.1205 USD per euro on 22 October 2026.
See the full calculation →
