Who will maintain local AI when IT is already stretched?

A long wait in the quotes department

The quotes department uses a local assistant to prepare summaries of requirements from documents. One morning employees wait a long time for answers, and for them the problem is a delayed quote. If the contract covers only the model server, a fault in the document module may fall outside the contractor's help.

The vLLM documentation describes metrics for waiting and running requests, response time and cache use. Such information lets the person who maintains the service see whether the problem lies in the model, the queue or the app. The team has to know where to report a problem and how to work during an outage, for example by going back to preparing the summary by hand.

What it costs and when it pays back

The count covers the time spent searching, reading and compiling material. What remains is the time to check the result against the source, 25% of today's work in the scenario. We measure both times in a trial on 20 to 100 cases the team has already handled.

The business signs off changes

Before an update, a person from the quotes department looks at the results on known tasks, for example three requirement summaries from the previous month. If the result got worse, the change waits and the service stays on the previous version. We write the hours for receiving reports, the way of responding and the limits of support into the maintenance contract.

Your own installation or the provider's environment

With an installation on site, the contractor takes over the agreed part of the upkeep, but someone is still responsible for the hardware, the network and service access. A private environment at a provider also moves the care of the infrastructure, if the data requirements allow it. The third point of reference is an approved cloud service, if it may process this data. The choice between a workstation and a server is covered in the article on a model on a workstation or a server.

Where to start

The first scope covers an assessment of one use and the current environment. The client names the owner of the task, the approved materials and what its IT can do. We deploy local models as part of private LLMs, and we build the view for working with sources as an AI app. For the free analysis, a description of one case and of where the materials sit is enough. Confidential files stay in the company, and you get a written plan within 2 working days.