When should you fine-tune AI to group complaints?

Three descriptions of a drawer

Suppose a furniture manufacturer receives these messages from three customers: “The drawer won’t close all the way,” “When I push the drawer in, it stays slightly open,” and “I can’t close the drawer, and the handle wobbles.” For its report, the quality team has agreed on two categories: “drawer closing problem” and “loose handle.”

The first two descriptions belong in the same group despite the different wording. The third contains both reported symptoms. If an employee can select only one category, either the handle problem or the closing problem disappears from the report. A model helping with this work needs to be able to assign more than one category to a complaint.

With this approach, three complaints contain three reports of closing trouble and one report of a loose handle. That makes four symptom occurrences, although there were three customers. The person reading the report needs to know what the numbers count. Otherwise, clearer grouping can still lead to a mistaken conclusion about the number of complaints.

These messages do not establish why the drawer will not close. A later product inspection may identify the cause. Keep the customer’s original description alongside the assigned categories, with inspection findings recorded separately. An analyst can then return to what was actually reported and compare it with what was subsequently found.

Agree on what belongs together

Useful grouping starts with a straightforward question for the team: what do we want to count together? If one employee labels every drawer problem “mechanism” while another separates opening and closing problems, their reports will not be comparable. Changing the model will not resolve that disagreement.

You do not need a detailed classification of every possible problem from the outset. Start with issues the team already discusses in quality meetings and can recognize in customer messages. A description such as “the drawer is broken” may warrant a general category or a request for more information, depending on the team’s process. A more specific category requires a more specific description.

Historical assignments can help, but they need review. They may reflect an earlier category system or include a technician’s findings that were absent from the original message. If the model will group customer descriptions alone, assess the examples on that basis. Otherwise, it is being asked to recognize information it has not received.

Do you need fine-tuning?

For a modest complaint volume, shared categories and manual tagging in the current system may be enough. If customers select a problem on a form, check whether those answers already provide the grouping you need. Rules that search for characteristic phrases can also help, although the drawer example shows why different descriptions of the same symptom matter.

Another option is an existing model given clear explanations of the categories. A trial on actual complaints will show whether it recognizes different wording and retains both problems in the third message. If employees have few corrections to make, additional training may not offer enough benefit to justify maintaining it.

Fine-tuning means training an existing model further using examples with the desired answers. Here, the team would teach it to apply its categories to the way customers describe problems. Google lists classification among the uses of this approach. It recommends improving instructions first, examining recurring errors, and ensuring that examples are labeled well.

Adaptation becomes worth considering when the team has consistent categories but an existing model still regularly confuses particular descriptions or misses a second symptom. It requires examples where employees can explain the correct assignment.

What would make the report better?

The comparison should use later complaints that were not part of the model’s training. A quality specialist can check whether the suggestions reflect what customers wrote and how much correction is needed before preparing the report. Compare that work with an existing model and the simpler method the team uses today.

A model may miss loose handles even while recognizing common drawer-closing problems correctly, especially when the customer mentions the handle as a second issue. The buyer should therefore see how it handles individual categories and messages containing several problems. In our example, omitting the handle would remove that symptom from the report entirely.

A new product or a change in complaint categories will call for another assessment. Agree on who reviews new mistakes and decides when categories should change. Grouping complaints for quality analysis is a separate task from choosing the team that should handle them. Our article on fine-tuning for support ticket routing covers that use case.

How Syntalith can help with the report

Syntalith builds AI applications and adapts models. For a quality team, we propose starting with a report that currently requires rereading complaints. We examine how the team groups them and compare simpler methods with an existing model. If recurring errors remain, a customization trial can show whether further model work is justified.

The selected approach should show employees the suggested categories alongside the original message, allowing them to correct an assignment before it enters the report. For an initial conversation, describe the report and a problem you find difficult to count today. You do not need a completed specification or customer records ready to share. We can agree on how to work with examples later; offer information is on Syntalith’s pricing page.