Fine-tuning, RAG or training your own model?

Two errors that look alike

The model answers by the old terms and conditions. Or it knows the terms but keeps sending complaints to the wrong department. To a user the two errors look alike, but the first concerns the source of knowledge and the second the way the model works. RAG joins search with generation: the app finds passages of documents and passes them to the model with the question, and the model's weights do not change. It works for terms, product sheets and procedures, but the result depends on the search engine, so you score retrieval and the answer separately.

In supervised tuning you show the input and the expected output: for a ticket, the department, the priority and the reason for escalation. LoRA trains extra matrices while the base weights stay frozen, and QLoRA combines this with a quantised base model, which limits the resources needed. Knowledge that has to be updated or deleted often is easier to control in a separate source than in the weights.

What fine-tuning or RAG on company data gives

In this count, RAG with the current terms gives most of the effect for a fraction of the cost. Training makes sense at a larger scale or when the remaining errors are expensive. According to Anthropic (2024), combining contextual embeddings, BM25 search and reranking cut failed retrievals by 67%. When the error lies in the source of knowledge, a change to retrieval like this usually gives more than tuning the model.

The set of training examples has to be maintained: when the process, the naming or the base model changes, the evaluation is repeated on the same test set. We put that cost in the quote together with the training itself.

When a small model is enough

Choosing one of a dozen or so departments can be handled by a classifier, an embedding model or a small decision model: you need a category, and free text only adds time. Jev-inspired models show the direction, and you compare them with a simple rule. Training from scratch means wider responsibility for data and the cost of computing, and it can make sense for a small, narrowly defined model, while building a general language model needs its own case. Related messages from one case stay in the same part of the split: five messages about one complaint spread across training and testing give an inflated result.

What the comparison should deliver

Before you commission the work, settle the cost of labelling, of comparing the variants and of a new evaluation after the base model is updated, because the LoRA technique alone does not set the price or the usefulness of a project. We adapt models as part of our private LLMs, and we build the app where an employee sees the result next to its source as part of our AI apps. For the free analysis, a few examples of an error that keeps coming back and a note of who corrects it today are enough.