Private LLMs: data, EU hosting and on-premise
Where should an LLM process company data? Compare on-premise servers, EU hosting and network isolation, including access, logs and GDPR review.
Syntalith
“Data stays with us” needs a precise definition. Does it mean inside the building, within the company cloud account, in an EU region or inaccessible to the model publisher? Each requirement leads to a different architecture and budget.
A private LLM lets you choose where inference runs. The application includes other components, however. Syntalith starts with a data map covering the input document, each service, the answer and the places where copies are created.
Three environments to consider
| Environment | When to evaluate it | What still needs to be organized |
|---|---|---|
| An on-premise server | Network constraints, existing hardware, local processing requirements | Administration, power, updates and recovery |
| A private deployment in an EU region | Hardware is difficult to operate or demand changes | Contracts, provider roles, locations of supporting services and backups |
| A disconnected environment | Work must continue without external connections | Dependencies, controlled file imports and updates |
On-premise describes location. An air gap describes network separation. They are different properties. An office server may send logs to a cloud service if monitoring is configured that way.
Follow every stage of the document
A scan passes through OCR, text goes into retrieval, passages reach the model and the answer returns to the application. Every stage may create temporary files and logs. An embedding model used for search can also run locally or through an external service.
We establish where these computations run and remove unnecessary content transfers. Agent tools need review too: a CRM request or web search is a separate data flow.
Enforce permissions before generating an answer
Sales staff and lawyers may use the same model while having access to different documents. Retrieval should respect the requesting user’s permissions. A prompt instruction does not replace authorization.
We also test role changes, revoked access and document deletion. Indexes, caches and conversation records may each need updating. Logging should support investigation without retaining every case’s complete contents indefinitely.
GDPR concerns the processing itself
The GDPR requires an appropriate processing basis, data minimization, retention limits and suitable safeguards. Depending on the relationship with a provider, processing arrangements need to be documented. Transfers outside the EEA have additional requirements.
Your data protection reviewers assess those questions. Syntalith supplies technical documentation, data flows and control configurations. Local weights or an EU server alone do not determine whether a specific workflow is compliant.
A model trained on confidential examples also needs protection. We restrict access to training data, adapters and weights. Evaluation includes checking for unwanted reproduction of training content.
What to establish before a proposal
Concrete requirements are useful: permitted regions, no internet connections during operation, user roles, retention periods and recovery expectations. A request for maximum security does not define any of these parameters.
A Syntalith private LLM deployment can cover the model, environment, integration and operations. Share requirements from IT or data protection reviewers. We can identify what the environment supports and what needs to be tested in a pilot.
Evaluate private AI for your organization
We help businesses and individuals select hardware, deploy a model and test it on their own tasks. Start with a computer you already own or ask us before buying one.
Private LLMs and fine-tuning