Fine-tuning for support ticket routing

What the model would actually learn

In this use, the model learns to predict the agreed category from the description of the case. It does not have to write a reply to the customer or solve the ticket on its own. A narrow scope makes benefits and mistakes easier to assess.

Google's documentation on supervised fine-tuning lists classification as one use of adaptation on labelled examples. It does not follow that every queue needs a trained LLM. The choice depends on the task and on a comparison with simpler options.

First check the categories and the data

Ask two experienced employees to assess the same hard cases independently. If they often differ, the rules of responsibility need clarifying. A model trained on inconsistent decisions will inherit part of that problem.

The ticket history also needs interpreting. The last team in the record need not be the correct first recipient. It may reflect a later escalation. Assess the data against the decision the system is meant to support.

Coverage of rare cases, product changes and the customers' language matters. The number of records alone does not settle whether you are ready to train. In Polish tickets, for example, allow for abbreviations, inflected names and Polish terms mixed with English ones.

Which solutions to compare

An adapted model should justify its extra cost of preparation and maintenance. If a simpler method gives a similar result, owning the model is no business benefit in itself.

Acceptance should count the cost of a mistake

Not all wrong assignments hurt equally. Passing a routine request to the neighbouring queue may cost a few minutes. Missing an urgent failure can delay the response much more. The criteria should reflect these differences.

Assess the categories separately, together with the cases the system hands to a person. Decide whether the result is a suggestion or triggers an action in the system. At the start it is easier to assess the quality of the proposals without changing the whole ticket workflow at the same time.

The final test should use independent cases that were not used to adapt the model. The buyer needs a comparison of the options on the same cases, a description of the limits and who takes over a ticket when the classification is uncertain.

What happens after launch

A new product, a reorganised service team or renamed categories can make the system work worse. The offer has to name the owner of the categories, how new errors are assessed and the scope of later changes. Training does not end the responsibility for quality.

The rights to use the data, how it is stored and the environment cleared for use are also part of the purchase. Do not assume that being able to export tickets means consent to use them freely for training.

How to talk to Syntalith

As part of AI applications and models, we can consider classification as part of a specific process. The first question is which error is worth reducing. A proposal should describe the options compared, the data, the acceptance criteria and responsibility after deployment. The details of preparing the data set and training stay part of the project work.

This article describes a use to assess. It claims no ready model and no effectiveness measured at a client. Pricing information is on the pricing page. For the call, prepare a description of the categories and a few anonymised cases that move between teams today.

What it costs

Where to start

In the free review we choose one process and 20 to 100 cases the team has already handled. We compare the simpler change with an adapted model on cases the model has not seen, and within 2 working days you receive a written summary with a price. We adapt models as part of private LLMs, and build the application as part of AI apps.