Blog · Published · Updated
What does a private LLM really cost?
Short answer
You compare a private LLM with an API on the cost of a correctly handled case. A Hetzner GEX45 GPU server with 8 hours of upkeep costs about €493 a month. A case with 3,000 input tokens and 600 output tokens costs about €0.0107 on the Claude Sonnet 5.5 API, so the server wins from about 58,600 cases. Every 500 extra corrections at 3 minutes each, however, cost €873.
How we workDetails ↓
- 01Call30 minutes with an engineer, free
- 02PrototypeA working system on your process and data, free
- 03ImplementationTarget and fixed price in the contract; 50% at signing
- 04Acceptance50% once the agreed target is met
- model on your server or in the cloud in the EU
- quoted after the review
- same work in application with EU cloud model
- from €3,000
- task, data and equipment analysis
- €0
We belong to the OpenAI and Anthropic partner programmes and learn through their official courses. What this means ↓
Author: Syntalith · 3 min read
Statistics
Local models in companies: statistics 2026
58,600
cases a month, from which a GPU server with upkeep at €493 comes out cheaper than the Claude Sonnet 5.5 API, at the same quality. Scenario.
20.0%
According to Eurostat (2025), 20.0% of EU companies with at least 10 employees used AI.
48.8%
According to Eurostat (2025), 48.8% of EU companies that considered AI and did not use it named concerns about data protection and privacy.
52.7%
According to Eurostat (2025), 52.7% of EU companies used paid cloud computing services.
11 of 13
According to the NoLiMa study (2025), at 32,000 tokens of context 11 of 13 models fall below half of the accuracy they reach with a short context.
Figures checked: 22 October 2026
What goes into the count
Base the estimate on the cost of a correctly handled case: an assigned ticket, a read invoice or an answer with a correct quotation. Set the unit before you compare models.
| Item | API | Your own model |
|---|---|---|
| Implementation | Integration, tests, error handling | The same plus server setup |
| Compute | Input, output and paid features | Buying or renting hardware, energy |
| Upkeep | Limits, API versions, monitoring | Also drivers, images and weight updates |
| Continuity | A plan for a vendor outage | Backups, recovery, spare capacity |
| Quality | Checking and corrections | Checking and corrections |
Small or irregular traffic
API
you pay for use
A large, regular volume of simple tasks
your own server
from about 58,600 cases
Data must not leave the company
your own server
whatever the threshold
Long documents and hard reasoning
API with a stronger model
quality before price
A model downloaded for free can handle a task at a higher cost than a paid API when it needs many corrections or an expensive server sits idle. The reverse happens too: a regular queue of simple tasks uses the hardware well. Fine-tuning adds data preparation, training and evaluation. You spread its cost over the period in which the tuned version works, and when the categories change every week, keeping the training set current costs more than one-off training.
The break-even point
- GEX45 server
- €214 + 8 h of upkeep × €34.90 = €493 a month
- A case on the Claude Sonnet 5.5 API
- $0.012 (€0.0107)
- Variable cost of a case on the server, energy above the base
- €0.0023
Threshold
€493 / (€0.0107 − €0.0023) = about 58,600 cases a month
Price of the GEX45 server with 24 GB of GPU memory from Hetzner's page on 22 October 2026, without the one-off €209 fee. €34.90 an hour is the EU average hourly labour cost for 2025 according to Eurostat. The API at Anthropic's rates, at the ECB rate of 1.1205 USD per euro on 22 October 2026. The variable cost, the upkeep time and the size of a case are the scenario's assumptions, at the same quality in both options.
Now add corrections. If the local model needs 500 extra corrections a month at 3 minutes each, that is 25 hours, which is €873. A small difference in quality can therefore reverse the result of the comparison. According to the NoLiMa study (2025), at 32,000 tokens of context 11 of 13 models fall below half of the accuracy they reach with a short context, so you test long documents separately.
Check when the tasks arrive
A monthly average hides the peaks. Documents arriving in the hour before the end of the day need different throughput from the same number spread over the night. We measure the length of inputs and answers, the queue and the waiting time. vLLM metrics cover requests and handling times, and the metrics documentation helps you gather real data.
- The number of cases a month and the peak hours
- A typical length of a document and of an answer
- The time of one correction
- Your current API bill, if you use one
- The specification of the computer or server you already have
More memory and the break-even point
Framework describes why more memory changes which model configurations you can run. That does not set the cost of a correct answer for your documents. Put the real hardware offer, its service life, energy, upkeep and the number of accepted cases into the formula, and compare the result with the cost of the OpenAI and Claude APIs.
Syntalith prepares a comparison of a local model and an API, measures the options and prices the implementation. In the estimate we keep the implementation work apart from hardware, hosting and care, so you know which part of the bill changes with traffic. Hardware details for Polish models are covered in the article on Bielik and PLLuM.
