How to assess AI skills before and after training

At Syntalith, we offer team training with an agreed way to assess participants' work. With the learning and development lead and the operational manager, we select tasks where staff will use AI. We agree what to assess at the start, what to teach, and how the recipient will judge whether the work is usable.

The trainer helps identify the difficulty: missing context supplied to the model, an overlooked source, or trouble assessing the response. That informs the practice and feedback on further attempts.

Start with the work the company wants to improve

For customer service, the task might be replying to a customer; for procurement, comparing bids; for operations, handing a case to a colleague. Each requires tool familiarity as well as an understanding of the purpose and reader. A general AI quiz will not show whether someone can prepare a comparison a buyer can use.

The program should cover the skills needed for the selected task. Syntalith's offer combines understanding a model's capabilities and limitations with providing context, analyzing documents, and preparing a response. It also covers data use in a company-approved tool and checking the result. Assessment should reveal which parts need more practice. If the difficulty is missing document access or an unclear company rule, the task result alone does not establish a skills gap.

The public introduction to CIPD's learning-evaluation factsheet, dated August 18, 2025 connects evaluation with identified performance gaps and business objectives. A training discussion should therefore include someone who knows the team's tasks and uses its work.

Establish the starting point, then try a new task

Before training, a short discussion about experience and a task resembling everyday work can help. “I use AI daily” describes frequency; completed work and an explanation of the choices show how the person uses the tool. An experienced specialist may be new to AI, while a confident user may skip checking content. These differences should inform the program, pace, and examples.

After training, use a new case of comparable difficulty. It should call for the same skills but contain different material, requiring the employee to select information and assess the response independently. Keep tool access, time, and the amount of material similar. If one example is short and complete while the other requires searching several systems, the difference in execution says little about a change in skills.

Suppose a team practices replying to a customer using an email thread. At the start, an employee writes a fluent message, but the reader cannot tell what the company needs from them. Training addresses the message's purpose, context for AI, and wording of the request. The later attempt concerns another customer and a different missing detail, so repeating the previous answer will not work.

What the manager should assess

Agree criteria that fit the task and discuss them before the work begins. In document comparison, accuracy against the sources matters, along with retaining differences that affect how a bid is understood. A message also needs a clear request and language suited to the reader. A handover needs to make the established facts and outstanding questions clear to the next person. Polished wording alone is insufficient.

Assessment also needs the basis for the answer. The employee should be able to explain the choice of material and identify the passages used. That helps distinguish a sound approach from a model happening to produce the right text. The trainer discusses specific gaps and helps decide what to practice again.

The L&D lead coordinates the program and collection of work, the manager contributes everyday job requirements, and the trainer selects tasks and provides feedback. Their agreement should make clear to employees what will be assessed. This approach supports learning and further assistance; it is not a basis for automated employment decisions.

What to commission alongside training

Before choosing a program, ask to discuss an example task and what the trainer would identify as needing revision. Establish whether the proposal includes assessing the starting point, independent work after training, and review of that work. Sessions, materials, and any return to the team after a period of practice are agreed in the proposal. Our guide to choosing an AI training provider covers supplier comparison in more detail.

Talk to Syntalith about training and assessing your team's work. Start with a task employees should be able to perform independently and who uses the result. We can discuss how to choose the learning activities and assess a new attempt. Further service information is available on our pricing page.