On-premises or air-gapped AI: choosing the required isolation

For companies processing confidential documents, the first requirement is often that documents must not leave the environment. A vendor declaration alone does not verify that boundary.

We built reproducible systems in two isolation regimes. Both make the boundary checkable with a command an auditor can run.

Both controlled cases use generated data and state their measurement scope. They contain no client data and provide no accreditation of a client's environment.

Level one: processing on site

The on-prem document workflow keeps the path from question to answer inside the client's environment. Semantic search, result ordering, the document index, and a local Polish-language model all run without an outbound network path. The controlled corpus contains 180 generated documents.

The product includes a test that attempts HTTP, HTTPS, DNS, and raw TCP connections from the model layer. None of the four controlled attempts connected. Search scored 0.958 for the ordering of the first ten results (nDCG@10) across 12 questions. Three generations by the local 4.5B model completed at a 95th-percentile time of 529.497 ms with PLN 0 in external API fees. The target 11B deployment needs its own hardware measurement.

Matching a Polish-language model to a Polish corpus is an important design decision. The selected model is openly available, licensed for commercial use, and offered in sizes that can fit client hardware.

Level two: a physically cut-off zone

Some environments go further and prohibit all outbound traffic. Delivery then becomes the central problem: the application and models must cross the zone boundary without opening an outbound path.

The air-gap release path answers with cryptography. Artifacts cross the boundary as a bundle with a SHA-256 manifest of every file and a signature over the whole. The recipient verifies the signature with a public key inside the zone; the private key never travels. A bad signature, a changed file, or an unresolved hash stops the release and has no bypass path, because a bypass path in a cut-off zone is a backdoor by definition.

The air-gap check accepted the correctly signed bundle, rejected the changed file, matched all four artifacts to the manifest, and produced a signed validation receipt. After a successful internal control, none of four outbound probes connected. The local model generation took 1.765 seconds, used up to 1.787 GB of memory, and incurred PLN 0 in external API fees. These technical checks do not certify a client's environment.

Which isolation to choose

Full physical isolation adds daily operational cost, from updates to diagnostics, and makes sense where a formal regime or a real threat model demands it. We begin with the threat model, formal requirements, and update path before selecting the isolation level.

If on-premises controls meet the requirement, an air gap may unnecessarily slow fixes and updates. If the regime requires physical separation, a firewall and vendor statement are insufficient.

Requirements shared by both regimes

Network isolation does not replace access control, retention, backups, or administrator security. Both designs need an inventory of models and libraries, artifact provenance, vulnerability scanning, separation of duties, and a tested recovery procedure.

Capacity and quality also need measurement. A local model may require more memory, and long documents require larger context. Before buying hardware, test representative documents for answer quality, time to first token, throughput, peak memory, and concurrent users.

The air-gap release lifecycle

The process needs explicit owners:

1. The build team prepares a frozen bundle and manifest of every file.
2. Security reviews dependencies, scans, and quality evidence.
3. A private key held outside the zone signs the bundle.
4. The zone operator verifies the signature and every hash with the public key.
5. Installation records the version, control result, and operator.
6. A rollback procedure restores the previously approved release.

The same discipline covers models, indexes, configuration, and data migrations. A new application with an old index may behave differently from the combination tested before crossing the boundary.

Failure modes and tradeoffs

An on-premises deployment often contains a hidden startup dependency that downloads a model, telemetry, or a font. Outbound probes should be supplemented by container inspection, DNS and firewall logs, and a restart test with no network.

Inside an air gap, the manual delivery path becomes the risk: an outdated public key, a missing artifact, an inconsistent migration, or pressure to bypass controls during an incident. The absence of an emergency bypass must be deliberate, and rollback needs rehearsal before release.

Counting total cost

External API fees may be zero while total cost includes servers, energy, observability, backups, administrator time, model updates, and release testing. An air gap adds bundle preparation, boundary control, media handling, diagnostics, and maintaining skills inside the zone.

Compare annual cost at the same user count, document volume, availability target, and update frequency. Security remains a hard requirement, so a lower price cannot reduce the required protection level.

Selection and acceptance checklist

• Does the requirement concern data location, outbound traffic, or physical separation?
• Who administers the system, and how are changes approved?
• How do models, libraries, configuration, and indexes cross the boundary?
• Does every artifact have a version, hash, provenance record, and signature?
• How are DNS, HTTP, HTTPS, and raw TCP tested?
• What are the backup, restore, and rollback procedures?
• How is quality measured on the target documents and hardware?
• How long does a security patch take in each regime?
• Can an auditor repeat the boundary check without the vendor?

Both case pages show the two regimes: the on-prem workflow and the zone release. A free scan of your threat model and isolation boundary is a good place to establish the required level.