Private AI starts with a boundary you can verify.

Updated October 6, 2026: expanded the deployment comparison, added a data-boundary worksheet and acceptance checks, and clarified the limits of private networking and local hardware. Original publication date retained.

Define what private means for this workflow

Start with a concrete requirement: which information must stay within which boundary, and who may operate the system? Keeping inference on premises, keeping an application in a company cloud account, and using a private network connection to a managed model are different designs. Choose against the requirement rather than treating “private AI” as a product label.

For a hypothetical internal document assistant, map the original files, extracted text, embeddings, retrieved passages, prompts, responses, logs, and backups. Identify every service that receives each item. A locally hosted chat interface does not establish where the model runs, and a local model does not establish where the rest of the application sends data.

A private endpoint is a network control

AWS documents interface VPC endpoints for connecting a VPC privately to Amazon Bedrock without requiring public IP addresses on the calling instances. The connection still targets Amazon Bedrock, a managed service. Our architectural conclusion is that a private path should not be described as a model running on your own premises.

The same documentation describes endpoint policies that control principals, actions, and resources, and notes that the default endpoint policy permits full access through the endpoint. Ask the implementer to demonstrate the intended identity and resource restrictions. Network placement alone is not an authorization test.

Keep networking, provider data handling, and application access as separate checks. A claim that data is not used for training does not, by itself, answer where it is processed, how long particular records remain, or who can retrieve them. Verify those details for the exact service and configuration proposed.

Review the whole path, including support and failures

List permitted outbound destinations and explain why each is needed. Include document parsing, embeddings, search, analytics, error reporting, model downloads, and backup storage. If the requirement is local-only operation, demonstrate the representative workflow with external access blocked, including its error paths. Record any exceptions rather than hiding them behind an overall label.

A successful network test is evidence for the tested version and conditions. Repeat the relevant checks when dependencies, tools, or configuration change. A failover route must respect the same data boundary; do not silently send a local-only request to a cloud model when the local service is unavailable.

Decide how an operator gets support without exporting private prompts or documents into an unrelated ticketing system. Define administrator access, approval for temporary access, log contents, retention, and removal of that access. Keep source permissions effective in search results, generated answers, and exports.

Size hardware around the workload

NVIDIA DGX Spark is one example of desktop-scale hardware for running models locally. NVIDIA’s product documentation describes local AI workloads; that does not establish the throughput, answer quality, or availability your application will achieve. Choose a model and test the actual workload before treating any hardware class as sufficient.

Measure concurrent users, input and output lengths, document processing, response time, and queue behavior using representative cases. Test after the system has been idle and while several requests compete for resources. Decide what users see when capacity is exhausted and how work continues during maintenance or hardware failure.

Check the license and deployment rights for the chosen model and supporting software. Open weights do not mean every model has identical terms or capabilities. Retrieval from approved documents may be sufficient for a knowledge assistant; fine-tuning is a separate decision that needs its own evidence and evaluation.

Assign operating responsibility and compare full costs

AWS’s shared-responsibility guidance distinguishes provider infrastructure duties from customer responsibilities, which vary by service. A client-controlled cloud account still requires ownership of permissions, application configuration, and the applicable maintenance work. On-premises operation also needs owners for equipment, updates, backups, and recovery.

Compare the same workload and service target across options. Include hardware or rented capacity, power, storage, networking, software, operations, spare capacity, and staff effort. Local hardware has capacity limits and replacement costs; hosted usage is not the only variable cost. There is no universal usage threshold at which one design becomes cheaper.

Copy this private AI deployment brief

# Private AI — deployment boundary
Workflow, users, and acceptable outputs:
Information that must remain within the boundary:
Boundary: premises / cloud account / approved provider:
Business, data, and technical owners:

## Data path
Documents and extracted text:
Embeddings and search index:
Prompts, retrieved passages, and responses:
Logs, analytics, support records, and backups:
Location, operator, retention, and deletion for each:
Permitted outbound destinations and reasons:

## Access and dependencies
User and administrator permissions:
Model and software licenses:
Model download, update, and telemetry behavior:
Remote support approval and expiry:
Provider processing and retention requirements:

## Demonstrate before acceptance
Representative quality and concurrency tests:
Denied user cannot retrieve restricted material:
Expected behavior with external access blocked:
Failure cannot trigger an unapproved cloud fallback:
Logs and backups stay within the agreed boundary:
Restore, update, and access-removal tests:
Observed exceptions and remediation owner:

## Operate
Capacity limit and overload behavior:
Maintenance and recovery owners:
Cost assumptions and comparison period:
Change triggers that require revalidation:
Acceptance reviewer and evidence location:

What DBAI helps scope

DBAI’s knowledge-systems service describes client-hosted deployment in a customer VPC or private cloud, alongside retrieval, source traceability, and permission-aware access. Start with the workflow and required boundary so the application, model, network, and operating responsibilities can be assessed together.

The acceptance decision should rest on a documented data path, representative output tests, permission checks, and a recovery demonstration. A hardware purchase or a private endpoint is one component of that evidence, not the complete deployment decision.