Private LLMs: data, EU hosting and on-premise

A server in the office, logs in the cloud

„The data stay with us” can mean the company's building, its cloud account, an EU region or a ban on sharing content with the model provider. Each answer leads to a different architecture and cost. On-premise describes a location and an air gap describes isolation from other networks, and the two are not interchangeable: a server in the office can send logs to a cloud service if monitoring is set up that way.

A document passes through text extraction, search, the model and the app, and each stage may leave temporary files and logs. The model that creates the document's representation for search can also run locally or as an external service. An agent's tools, such as a call to the CRM or an internet search, are separate data flows.

What it costs and when it pays back

The count covers the time spent searching, reading and compiling material. What remains is the time to check the result against the source, 25% of today's work in the scenario. We measure both times in a trial on 20 to 100 cases the team has already handled.

Permissions must work before the answer

A salesperson and a lawyer may use the same model but see different documents, so the app limits search by the user's permissions. An instruction in the prompt does not replace access control. We also test a change of role, the removal of permissions and the deletion of a document, because the index, the cache and the chat record may each need a separate update. A model fine-tuned on confidential examples needs protection too: we limit access to the training set, the adapters and the weights.

When EU hosting is enough

If the requirement is an EU region and a data processing agreement, a service in a European cloud region often costs less than your own server. When the requirement forbids internet connections during operation, you need a server on site or an isolated network. The GDPR requires a legal basis for processing, data minimisation, limited retention and security measures, and processing outside the EEA is subject to the transfer rules. The person responsible for data protection assesses these questions, and Syntalith provides the technical description, the data flow and the configuration. Signing off an environment with no connection is described in a local model in manufacturing.

Where to start

The most useful inputs are specific requirements: the allowed region, no connections during operation, user roles, the retention period and recovery after a failure. The phrase „the highest security” sets none of these parameters. We deploy local models as part of private LLMs, and we build the view for working with sources as an AI app. For the free analysis, a description of one case and of where the materials sit is enough. Confidential files stay in the company, and you get a written plan within 2 working days.