Blog · Published · Updated
Assessing the business cost of custom model errors
Short answer
Two models can make mistakes equally often and leave a team with very different work. A false alarm costs a few minutes of reading, and a missed problem costs half an hour of clarification after fulfilment has started. At 10,000 orders a month a more cautious threshold gives 100 more alarms, but 40 fewer misses, and saves 15 hours, which is €524.
How we workDetails ↓
- 01Call30 minutes with an engineer, free
- 02PrototypeA working system on your process and data, free
- 03ImplementationTarget and fixed price in the contract; 50% at signing
- 04Acceptance50% once the agreed target is met
- model on your server or in the cloud in the EU
- quoted after the review
- same work in application with EU cloud model
- from €3,000
- task, data and equipment analysis
- €0
We belong to the OpenAI and Anthropic partner programmes and learn through their official courses. What this means ↓
Author: Syntalith · 3 min read
Statistics
AI model quality: statistics 2026
15 h
a month saved by a more cautious threshold at 10,000 orders: 75 hours of work on errors in place of 90, despite more alarms. Scenario.
90%
According to the European Commission (2026), 90% of EU companies that use AI say people take the decisions.
Source: European Commission, Flash Eurobarometer on AI (2026) ↗
40%
According to Gartner (2025), over 40% of agentic AI projects will be cancelled by the end of 2027 because of costs, unclear value or weak risk controls.
25.1%
According to a Harvard Business School study (2023) of 758 consultants, tasks within the model's capabilities were done 25.1% faster with AI.
14–56%
According to Anthropic (2025), AI cut task time by 14 to 56% in studies with a control group.
Source: Anthropic, Estimating AI productivity gains (2025) ↗
Figures checked: 22 October 2026
The same error at a different point in the process
The model flags orders whose delivery notes need clarifying. A false alarm means an employee opens a correct order, reads the note and closes the case. A miss means going back to the document after shipping has started, contacting the customer and changing the work plan.
False alarm
3 min
The employee reads a correct order and closes the note.
Miss
30 min
Going back after fulfilment has started, a call to the customer, a change of plan.
Not every miss has the same consequences. If the employee reads all the notes before fulfilment anyway, they catch the problem in the normal course of work. When the model's flag decides which notes anyone reads at all, the solution carries more responsibility. Assessing it therefore needs knowledge of the place where the company wants to use it.
In the description of an error, keep what happened to the case next. How many people had to go back to it? Could the work wait? Was the correction easy, or did completed steps have to be undone? The answers separate extra reading from a disruption to fulfilment.
More flags mean more checking
The model can be set more cautiously, sending more cases to a person. More frequent flags reduce misses, but take up the reviewers' time. The scikit-learn documentation separates scoring a classification from the decision taken on it, and explains how the threshold changes that decision.
The cost of errors at two thresholds
- Starting threshold
- 800 alarms × 3 min + 100 misses × 30 min = 90 h
- More cautious threshold
- 900 alarms × 3 min + 60 misses × 30 min = 75 h
- Difference
- 15 h (€524) a month
Work on errors falls by 16.7%
The error counts and the times are the scenario's assumptions. The result shows the cost of work, without delays, penalties or lost customers. €34.90 an hour is the EU average hourly labour cost for 2025 according to Eurostat. Add the cost of changing the threshold and of upkeep separately.
More alarms make sense only while their queue can still be handled. If the review takes longer, people start reading important flags in a hurry, and the number of problems detected no longer describes the workload.
Simpler fixes first
The ambiguity comes from a missing field
an extra field in the form
the cheapest fix
A repeated pattern in the data
a simple rule before the model
a small scope
Mistakes in understanding the content
fine-tuning the model on checked examples
after an assessment
The extra checking eats the benefit
a full review by an employee
no model
The review also covers the current way of working. A simple rule or a form field can remove part of the ambiguity before you use a model at all.
Which decision to commission
- 01ConsequencesThe client's team describes the consequences of chosen errors and who catches them today.
- 02ResultsSyntalith lines up the models' results on the same cases.
- 03ThresholdsA comparison of the cost of work at several thresholds.
- 04DecisionA model correction, a form change or a full review.
The outcome may be a narrower use: the model sorts the queue, and an employee still reads all notes before fulfilment. It may also turn out that the extra checking swallows the benefit of automation. Both findings are useful before you order further training of the model. For a first conversation about assessing a private model, a description of the mistake that caused the most returns to work is enough. Server costs are covered in the article on the cost of a private LLM.
