Blog · Published · Updated
How to compare AI agents for a business
Short answer
You compare AI agents on one process and the same set of cases, with the expected result in the system written down before the demo. What counts is the operation performed, the behaviour when data is missing or access is refused, and the handover to a person. At Syntalith an agent costs from €3,000 net, and with 500 customer cases a month at 8 minutes each it pays back in 1.7 months.
How we workDetails ↓
- 01Call30 minutes with an engineer, free
- 02PrototypeA working system on your process and data, free
- 03ImplementationTarget and fixed price in the contract; 50% at signing
- 04Acceptance50% once the agreed target is met
- net per agent for one process, paid once
- from €3,000
- from signing the contract to acceptance
- 6–16 weeks
- free prototype on your cases
- €0
We belong to the OpenAI and Anthropic partner programmes and learn through their official courses. What this means ↓
Author: Syntalith · 3 min read
Statistics
AI agents: statistics 2026
1.7 mo.
payback of a €3,000 agent at 500 customer cases a month: €2,327 of work today, €599 with checking and the model. Scenario.
130
According to Gartner (2025), of the thousands of agentic AI vendors only about 130 offer real agentic solutions.
40%
According to Gartner (2025), over 40% of agentic AI projects will be cancelled by the end of 2027 because of costs, unclear value or weak risk controls.
50%
According to Anthropic (2026), software engineering accounted for nearly 50% of agent activity on the public API, and most agent actions were low-risk and reversible.
Source: Anthropic, Measuring AI agent autonomy in practice (2026) ↗
90%
According to the European Commission (2026), 90% of companies using AI say people take the decisions, and 2% of decisions are fully automated.
Source: European Commission, Flash Eurobarometer 578 (2026) ↗
Figures checked: 22 October 2026
Write down the expected result before the demo
A ranking of agents makes sense only once you name the work. A research assistant and a system that updates orders can use the same model and differ in everything that matters to the company. So you choose one process and prepare the same material for every solution you compare. For each case you add what should appear in the target system and when the agent should stop.
In a sample task a client asks to change the delivery address, and the order has already gone to dispatch. The agent should recognise the constraint, leave the address in the order unchanged and prepare the case for the person who can contact the carrier.
| Trial | What the buyer checks |
|---|---|
| An ordinary request | whether the result concerns the right case |
| A missing order number | whether the agent asks for the missing information |
| Refused access | whether it avoids trying the operation another way |
| A lost confirmation | whether it establishes the result of the previous write before retrying |
| Handover to an employee | whether the person sees the context and the reason for the stop |
- 01Inputthe same client message and order status for every agent
- 02Expected resultaddress unchanged, case with the person who handles transport
- 03Runa record of tool calls and of changes in the system
- 04Assessmentthe operation, the text and the handover scored separately
- Successful and rejected runs, without picking the best ones
- Wrong actions kept apart from answers that need language edits
- Handling time, including preparing the material and checking
- Data export, the history of actions and how maintenance is handed over
What an AI agent costs and when it pays back
- Today
- 500 customer cases about orders × 8 min = 66.7 h × €34.90 = €2,327
- With the system, a person checks the output in 25% of the time
- €582
- Model, 3 calls per case, Claude Sonnet 5.5
- €17
- Cost of one case
- €4.65 today, €1.20 with the system
- Saving
- €1,728 a month
Payback of €3,000
1.7 months
The volume, the time and the checking share are the scenario's assumptions. €34.90 an hour is the EU average hourly labour cost for 2025 according to Eurostat. Model: 4,000 input tokens and 500 output tokens per call at Anthropic's rates, converted at the ECB rate of 1.1205 USD per euro on 22 October 2026.
The count covers the work around a case: searching, reading, writing and passing it on. What stays is the time to check the output, 25% of today's work in the scenario. Before we quote, we measure both times on 20 to 100 cases your team has already handled, in a €0 test that takes 5 working days.
According to Anthropic (2026), software engineering accounted for nearly 50% of agent activity on the public API, and most actions were low-risk and reversible. Changing a client's order is harder to reverse than changing code in a branch. That is why the trials with refused access and a lost confirmation carry the most weight in the comparison.
The name of a library is no assessment of the solution
The LangGraph documentation describes durable state, resumption and human involvement. These are building blocks for an application, and having them does not confirm that a specific implementation is correct. A retry needs an assessment of the integration: two answers that sound right can mean two writes in the system. According to Gartner (2025), of the thousands of agentic AI vendors only about 130 offer real agentic solutions. A test on your own cases separates agents from chatbots with a new name. How developers choose a library is described in how to fairly benchmark agent frameworks.
Every candidate performs the operations correctly
maintenance, data export and cost decide
priced after the review
A candidate writes well and gets operations wrong
reject it, or a scope without writes to the system
from €3,000
No ready-made agent handles the exceptions of the process
an agent built for the process
€3,000 to €17,500
Where to start
The first comparison covers 20 to 30 cases from one process, including the five hard trials described above. The set stays with you and serves to accept every later version. We build this kind of rollout as part of AI agents. For the free process review, a description of one case from recent weeks and the tools it went through is enough. You do not need to send confidential documents, and you get a written plan within 2 working days.
Worked example · Scenario
How we count the payback: online shop customer cases
We run the same count for your process in the free process review.
€2,792
a month for handling 800 orders that bypass the checkout
Payback: 1.3 mo.
6 minutes to retype an order or answer a question and checking in 15% of the old time are our assumptions. €34.90 an hour is the EU average hourly labour cost for 2025 according to Eurostat, for companies with 10 or more employees. Model cost: reading the order in our system (about €0.0006) plus matching the lines (Claude Sonnet 5.5, about 3,000 + 500 tokens), at the ECB rate of 1.1205 USD per euro on 22 October 2026.
See the full calculation →
