What does a private LLM really cost?

A freely downloadable model can cost more per task than a paid API. It may need frequent corrections, or its server may sit idle. The opposite can also happen: a steady queue of simple tasks can use hardware well and reduce external inference charges.

A private LLM assessment should therefore use cost per correctly completed case. That might mean a routed ticket, an extracted invoice or an answer with a valid citation. Define the unit before comparing models.

What belongs in the calculation

Fine-tuning adds data preparation, training and evaluation. Spread that cost over the period in which the adapted model actually runs. If categories change weekly, maintaining the dataset can cost more than the initial training job.

A worked break-even scenario

The following figures are illustrative assumptions. They are neither Syntalith prices nor measured deployment results. PLN is used to make the example directly usable for a Polish budget.

Assume monthly fixed local costs F = PLN 2,400, variable cost per case v = PLN 0.03 and an API cost a = PLN 0.15 for the same case. Quality differences are temporarily excluded and added afterwards.

local cost = F + N × v
API cost = N × a
break-even volume N = F / (a - v)
N = 2400 / (0.15 - 0.03) = 20,000 cases per month

At 10,000 cases, local operation costs PLN 2,700 and the API costs PLN 1,500 in this scenario. At 40,000 cases, they cost PLN 3,600 and PLN 6,000 respectively. If a is no greater than v, increased volume cannot recover positive fixed costs.

Now include corrections. An additional 500 cases requiring three minutes each create 25 hours of work. At an assumed PLN 80 per hour, that adds PLN 2,000. A relatively small quality difference can reverse the result.

Check when requests arrive

A monthly average hides peaks. Documents arriving in the hour before closing need different capacity from the same volume spread across the night. We measure input and output lengths, queues and user waiting time. vLLM metrics include request and timing information; the applicable measures depend on the deployed version.

Quantization can reduce weight memory. Longer contexts and concurrent users consume some of the space recovered. A smaller model adapted to a narrow task may also avoid the cost of a large generator. Both options need the same quality assessment.

What to bring to a cost assessment

Prepare monthly volume, peak times, typical document length and the time needed to correct an error. Add current API bills where available, plus hardware specifications. Aggregated figures and a task description are enough for the first conversation.

Syntalith can compare local models with an API, measure the candidate configurations and quote deployment. We separate implementation from hardware, hosting and ongoing support so you can see which costs change with demand.